“Neural networks represent the beginning of a fundamental shift in how we write software. They are Software 2.0.”
Andrej Karpathy
The current coding paradigms nudge developers to write code using restrictive machine learning libraries that can learn, or explicitly programmed to do a specific job. But, we are witnessing a tectonic shift towards automation even in the coding department. So far, code was used to automate jobs now there is a requirement for code that can write itself adapting to various jobs. This is software 2.0 where software writes on its own and thanks to machine learning; this is now a reality. Differentiable programming especially, believes the AI team at Facebook, is key to building tools that can help build ML tools.
To enable this, the team has picked Kotlin language. Kotlin was developed by JetBrains and is popular with the Android developers. Its rise in popularity is a close second to Swift. Kotlin has many similarities with Python syntax, and it was designed as a substitute for Java. In the next section, we look at how Facebook is appending new capabilities to Kotlin so as to build next-generation software toolkits.
Overview Of Software 2.0
Software 1.0 or the software as we know it comprises the usual languages such as Python, C++, etc. The programmer writes explicit instructions to the computer; instructions that tell the computer how to behave. In contrast, wrote Andrej Karpathy, the director of AI at Tesla Motors, in one his blogs, that Software 2.0 can be written in much more abstract, human-unfriendly language, such as the weights of a neural network. Machine learning fits perfectly into the software 2.0 transition that’s happening right now.
For instance, computer vision applications, wrote Karpathy, require engineered features with a bit of machine learning, and now with the help of large datasets like ImageNet, researchers have developed techniques that search the space of neural network architectures itself, which is an active area of research currently. So, it is established that AI and automated software development cannot be separated. And, Facebook’s AI team in an effort to leverage this capability to build better ML programming tools bringing in Kotlin programming language into the mix.
How Kotlin Is Being Used
“Facebook AI is building an automatic differentiation system for the Kotlin language.”
The team at FB, incorporating the concepts of differentiable programming to improve the capabilities of Kotlin, which in turn can be used to build new toolkits. In differentiable programming, said Facebook, library code could be incorporated into more comprehensive models, and it also allows developers to leverage gradients to automatically optimise parameterised programs that aren’t written using machine learning libraries.
The intuitive nature of differentiable programming in Kotlin allows developers to create programs that are flexible and take advantage of the structure of the problem while keeping debugging simple.
According to the team at Facebook AI, Kotlin enables Software 2.0 through:
- Automatic differentiation.
- Tensor typing.
- Generating compile-time errors for differentiable functions and tensor shapes.
- Making available a library that provides a Tensor class and machine learning APIs.
“We’re extending the Kotlin compiler to make differentiability a first-class feature of the Kotlin language. Our work enables developers to explore Software 2.0, where software essentially writes itself.”
Facebook AI
Facebook announced that they are building an automatic differentiation system for the Kotlin language. Automatic differentiation refers to constructing a procedure for computing derivatives. An automatic differentiation system dissects any given program into its primitive operations(e.g., add, subtract etc.) to compute derivatives.
The Facebook team has also integrated Kotlin language with the IntelliJ IDE extensions so that developers can get real-time feedback. As shown above, a simple convolutional neural network written in IntelliJ where the developer can inspect the resulting shapes at each step.
Very soon, Facebook will also be releasing a user library that packs the advantages of automatic differentiation and so that the developers can use these independent of the framework they are working on.
Know more about this project here.
