Does democratisation of machine learning work for everyone or is there a flip side to it? What do the developers with no machine learning background think about it?
Ever since AI-based jobs have been crowned the hottest job of this decade, there has been a rush amongst across many industries to transition into an ML engineer or an AI specialist. This rush was fueled by the introduction of high-level interfaces that allowed the developers to build ML pipelines and deploy with very little knowledge of the underlying concepts.
Have the high-level APIs and frameworks such as TensorFlow made their life easier or did they create a disconnect within the ecosystem?
To answer these suspicions, a team from Google Brain in collaboration with UC San Diego have conducted a survey that included developers who are working on Java and have made a switch to ML roles. There have been many successful transitions into new data-driven roles but, the question that one would like to know is — are these developers happy? Well, not so much, says the survey done by Carrie Cai of Google Brain and Philip Guo of UC San Diego.
How Useful Are APIs & Frameworks
To investigate the motivations, hurdles, and desires of the software developers, the researchers have deployed a survey on TensorFlow.js ML framework and gathered 645 responses.
Here are a few of their findings:
Lack of fundamental understanding
In the survey, the most frequently mentioned hurdle, according to the authors, is the lack of conceptual understanding. Though generally skilled in programming, respondents often encountered a “steep learning curve of learning the foundation of ML concepts,” and felt they “never get a full understanding of the algorithms.”
Many also felt that they lacked “linear algebra knowledge,” “good calculus background,” or “statistical experience.” Respondents described being “too often faced with mathematical equations (e.g. backward propagation), that are very hard to implement correctly.”
Beginning with the ‘hello world’ of machine learning to a few tutorials later is hard because of the lack of statistical experience and math behind it
Thus, even in cases where logistical barriers are reduced (e.g., easier setup in web-based environments), some still perceived a steep learning curve due to math-related concepts.
No matter how high-level machine learning APIs are, a practitioner usually would falter or would feel detached from their work when they lack the fundamental understanding of algorithms.
Challenges during Implementation
As tutorials tend to offer a limited set of simple examples, wrote the authors, people struggled to identify the correct architecture to use for their specific scenario. As a result, many non-ML developers fumbled with implementation. An implementation can be something as fundamental as deciding the number of units needed while adding layers or finding the suitable activation functions or loss functions.
Faced with ample options, developers may simply turn to the most popular or convenient frameworks with the lowest perceived overhead.
Because ML relies on training data at its core as finding and cleaning interesting yet simple datasets could be challenging too.
So What Can Be Done?
While some frameworks do offer pre-made machine learning modules, these resources could be more useful to non-specialists. However, only if they came equipped with best practices for how to adapt them to example problems so that users would know when and where they can be applied.
Addressing the challenges listed in the survey, Google AI has come with a blog post that has shed light on some of the strategies to be imbibed for a smoother adoption of ML workflows.
Their recommendations can be summarised as follows:
- Embed implementation tips into the programming workflows. Ex: TensorBoard
- Just-in-time strategic pointers to learn the underlying math behind machine learning. For instance, a pointer can be a tip that says if one has to build a model from scratch or use a pre-trained model.
- Get more hands-on with code demos that assists the rookie to run and edit every small chunk with better understanding.
- Popular frameworks should have tutorials that don’t just stop at ‘HELLO WORLD’.
Though TensorFlow is popular amongst the developers for making their ML chores easy, there is still this looming question of explainability in AI, which is a concern for people on both ends— non-ML developers and customers.
So, TensorFlow’s team has come with TensorBoard. With TensorBoard, one can accomplish the following:
- Tracking and visualizing metrics such as loss and accuracy
- Visualize the layers
- Check how weights, biases change over time
- Displaying images, text, and audio data
According to a survey done on emerging jobs of 2020 by Linkedin, AI-based roles have witnessed a 74% annual growth in the past four years. There is no doubt that these jobs require a decent understanding of machine learning algorithms, frameworks like TensorFlow and programming languages such as Python.
Since the efforts to lower the barrier for easier adoption also comes with hurdles for the non-ML developers, there is an immediate need for rethinking the design strategies of these human-machine interfaces.