Open-source software (OSS) underpins all major cloud platforms and a large number of cloud-based services today. Core technologies that power cloud computing –Linux, Kubernetes, Cloud Foundry, Docker, to name a few –are developed in open communities. The Cloud Native Computing Foundation supports innovation in OSS for cloud computing and has sponsorship and participation from practically all major players in the cloud ecosystem.
A recent survey  of 3,440 professional developers and managers showed the widespread influence of open source, both in cloud technology and as an essential in-demand and growing skill in the industry. For example, 69% of respondents thought that contributions to open-source projects result in better professional opportunities. These are all indicators of the widespread trend of OSS becoming a dominant place for innovation in cloud platforms, services, and applications. Moreover, this innovation happens in an open, community-driven paradigm, wherein these communities operate on the fundamental principle of open collaboration via open sharing and the development of source code.
Academia & research labs as custodians of innovation
Another set of communities for sustained, ground-breaking innovation for the past decades have been academic and corporate research laboratories (labs) and who are often seen collaborating. These labs have researchers and students more oriented towards advancing state-of-the-art technologies using rigorous scientific methods such as reproducible experimental validation and peer review. In addition, such labs openly share the outcomes and innovations of their pursuit of scientific and technological advances via research conferences, publications in journals and build on top of each other’s work.
Traditionally these two sets of communities – OSS communities and research labs – have been distinct, with very different structures to collaborate. There have been areas where these communities have intersected, but that is only in relatively limited areas and from select research labs. However, we are also witnessing a change where these labs release libraries or utilities under open source licences to accelerate innovation and wider adoption.
Unlocking the potential of community-driven innovation
Community-driven research has been on the rise, and there have been some instances in recent years across various areas like health, climate, AI, etc. An example is the “papers with code” trend that has emerged in Artificial Intelligence (AI) research, where the research work gets published along with releasing the associate source code as open source. TensorFlow and PyTorch are two popular open-source projects, where the technology and models developed thereon have been published in research communities and shared via open source. This combination of publication and open source has become a broadly accepted and expected way of sharing work in the AI research communities. It has led to tremendous acceleration in research and innovation. Researchers can access not just a description of another researcher’s work but also the source code, which allows quick reproduction, benchmarking, and improvements.
A similar unification of open source and traditional research collaboration can be leveraged in many other areas of Computer Science and Engineering. With the advent of cloud-based collaboration tools such as GitHub, Slack, CI/CD (continuous integration / continuous deployment) tools, and container-based packaging for easy reproducibility of runtime environments, this trend will only accelerate. These tools enable researchers to easily reproduce the exact environment of any other researcher’s work. Historically, this has been a challenge since much effort is spent in reproducing published results. Now, with the ability to package a full environment – including the operating system, versions of software and their configuration – as Linux container images and transfer them easily, this becomes as easy as running a single command on the command line. There are already projects such as Rocker and Binder, making this even easier for data science researchers.
The next era of cloud computing innovation
As research labs across academia, industry, and government leverage these collaboration mechanisms, we can expect a tremendous acceleration of research on ground-breaking innovations similar to open communities. In all likelihood, the developer and citizen communities will work in close partnership with traditionally siloed research labs, driven by the organic alignment of common interests and skills.
The next age of innovation in cloud-based computing will be driven – in fact, further accelerated – by coming together of different communities. An example of this trend in the Hybrid Cloud space is the Konveyor project that brings together a global community of researchers and developers to create new tools for migrating and modernizing legacy applications to Kubernetes. Developing such tools will require a combination of fundamental research in AI for Code, program analysis, and distributed computing, coupled with the agile and disciplined software development principles of open source. Such projects will create the opportunity for bringing the scientific method to the open-source community and the community-driven, participatory innovation model to the research labs.
We are at an inflexion point in the evolution of the scientific research method – from an era when it was largely restricted to the confines of research labs to an era when research labs and open communities work hand in hand for accelerated discovery, invention, and translation to practice.
References Andy Oram, “The Value of Open Source in the Cloud Era”, O’Reilly Media, 2021.
Subscribe to our NewsletterGet the latest updates and relevant offers by sharing your email.
Amith Singhee is a Senior Manager and Senior Technical Staff Member at IBM Research, India. He leads the Hybrid Cloud research mission for IBM India, where he is responsible for developing innovations that help IBM’s enterprise clients effectively adopt Hybrid Cloud. He has an extensive experience of over 13 years in leading high-value research projects and teams to successfully develop and deliver value through fundamentally new solutions, platforms, algorithms and models for Hybrid Cloud, Retail, Building, Agriculture, Smarter Energy and Electronic Design Automation. Amith has a B.Tech in Electrical Engg. from IIT Kharagpur, with a PhD. in Electrical and Computer Engg from Carnegie Mellon University.