Using Kubeflow to solve natural language processing problems

Natural Language Processing (NLP) is a set of techniques and algorithms that enable computers to read, understand, and interpret human languages. NLP is a necessary discipline for any developer looking to build robust chatbots, speech recognition or real-time language translation software. The main difficulty for NLP developers is that computers are designed to understand programming languages, which are explicit and well-structured. Whereas the natural language humans use is neither explicit nor well-structured! 

Because of these issues, machines have historically struggled to understand the context of natural language. Furthermore, developing a complicated set of rules may only be effective for a restricted or narrow number of problems. Fortunately, due to recent advances in computing power, algorithms and machine learning software, computers can now more effectively “cope” with the ambiguity that human language often presents.

In the sections that follow we’ll look at a few real-world NLP use cases in greater detail.

Use Case: Automated Speech/Voice Recognition

Speech recognition software, also referred to as automatic speech recognition (ASR) or speech to text (STT), translates human speech from its analog form (acoustic sound waves) into a digital form that machines can recognize and then operate on.

Below is an example of what a typical ASR workflow looks like.

Now, let’s take a look at what happens in each step of the ASR workflow.

• The first thing to do is split the audio files of a speech recording into “tokens” or individual sounds. These tokens are essentially small chunks of the audio files that are typically in the range of 6 – 10 seconds in duration.
• Next, we need to analyze each sound in the context of speech.
• We can use different types of algorithms like NLP, deep learning Hidden Markov Model, or N-grams to find the most probable word fit in that language model.
• At this point we perform “speech to text” by converting each speech audio into text.
• For example, Handover to NLU is useful for defining a meaning to each of the text adapted models.
• An ASR model output example may look like the following:

r  eh k ao g n ay  z   s  p  iy  ch   =   “recognize speech”

r  eh k ay n  ay s b  iy  ch   =   “wreck a nice beach”

Use Case: Chatbots

Every company wants to provide excellent customer service by having the capacity to respond 24/7 to customer inquiries. In reality, however, this can be either impractical given staffing levels and/or expensive. Enter virtual assistants and chatbots!

Ideally, chatbots are able to interact with people in the same manner that a human would. The more “human” and less “robotic” the interaction, the better the experience will be for a customer. To achieve this level of sophistication, the bots require the effective use of natural language generation (NLG) and advanced natural processing capabilities.

Here are some of the benefits NLG and NLP bring to chatbot development.

• With adequate understanding, there are fewer false positive outcomes
• By leveraging statistical modelling, it can recognise user input failures and resolve conflicts
• For user responses, it can employ more comprehensive communication
• Learn more quickly and close development gaps in the process
• Reduce the amount of training data required to achieve natural language capabilities
• The ability to reconfigure input training data for future learnings
• Simplified corrective measures are provided for false positives

Now, let’s take a look at how chatbots can make practical use of NLP.

Natural conversations across languages

The problem with a pre-fed static content approach to bots is that languages have an infinite number of ways to express a factual claim. There are also an infinite number of ways for a user to create a statement to articulate emotion. Fortunately, in recent years researchers have made tremendous gains in how systems interpret human languages. It is now possible to link the incoming text from a human with a system-generated response using NLP. This response can range from a simple answer to a query to an action based on a client request, or the storage of any information from the customer in the system database.

Because NLP-powered chatbots are capable of comprehending language semantics, text structures, and speech phrases, they can also make sense of vast amounts of unstructured data. For example;

• NLP can understand morphemes across languages, making a bot more proficient in identifying various nuances.
• NLP enables chatbots to read and interpret slang, learn abbreviations, and understand different emotions across sentiment analysis, just like humans.

Help staff focus on mission critical tasks

When organizations deploy NLP-based chatbots, for example in Human Resources or IT Helpdesk functions, they help reduce repetitive tasks/communications. This allows the staff in those departments to focus on more mission-critical activities.

Higher Customer Satisfaction

People these days expect instant responses and solutions to their questions. NLP enables chatbots to understand, analyse, and prioritise questions based on their complexity, allowing bots to respond to customer queries faster than a human. Faster responses aid in the development of customer trust and, as a result, more business. When you use chatbots, you will see an increase in customer retention. It reduces the time and cost of acquiring a new customer by increasing the loyalty of existing ones. 

Market Research and Analysis

Social media alone can provide or generate a significant amount of versatile and unstructured content. NLP aids in the framing and interpretation of unstructured content. You can quickly grasp the meaning or concept underlying the customer reviews, inputs, comments, or queries. You can get a sense of how the user feels about your services or brand without having to ask them directly.

Use Case: Machine Translation

Machine Translation, also known as “robotized interpretation”, is a process in which a computer program interprets text from one language into another without human intervention. Machine translation, at its most basic level, is the simple replacement of atomic words in one distinctive language with words of another.

More intricate translations can be carried out using corpus approaches, taking into consideration the improved treatment of phonetic typology contrasts, express acknowledgement, and idiom translations, as well as the seclusion of peculiarities. Although most systems today are unable to perform as well as a human translator, the gap is closing quickly. One only has to think about the world of chess and what happened with Deep Blue and other programs.

One of the most important advantages of machine translation is speed. As you may have noticed, computer systems can quickly translate large amounts of text. Because it is less expensive than engaging a human translator, machine translation provides the right balance of speed, accuracy and cost. Another advantage of machine translation is that it may learn significant terms and reuse them in new contexts whenever you need them.

Next we’ll cover the four types of Machine Translation to be aware of.

Statistical Machine Translation

Statistical Machine Translation (SMT) operates by referencing statistical models that rely on the analysis of massive amounts of bilingual information. It anticipates determining the relationship between a source language word and a target language word. Google Translate is a great example of this.

Rule-based Machine Translation

Rule-based Machine Translation (RBMT) translates the fundamentals of grammatical rules. To construct the translated sentence leads to a grammatical evaluation of the source and target languages. However, RBMT necessitates extensive editing, and its heavy reliance on dictionaries suggests that proficiency is attained only after a long period of time.

Hybrid Machine Translation

Hybrid Machine Translation (HMT) is a hybrid between RBMT and SMT, as the name implies. It makes use of translation memory, which makes it undeniably more successful in terms of quality. Nonetheless, HMT has a number of drawbacks, the most significant of which is the need for extensive editing, as well as the use of human translators.

Neural Machine Translation

Neural Machine Translation (NMT) is a sort of machine translation in which statistical models are built using neural network models with the end objective of translation. The main benefit of NMT is that it provides a single system that can be configured to unravel both the source and target text. 

Using Kubeflow to Accelerate NLP to Production

To bring an NLP use case to production requires a ton of manual work to be performed by a variety of data scientists, systems, SecOps and data engineers. It is found in one research study that one ASR pipeline model requires around 6 months before it can be deployed to production! The way to reduce the time it takes to bring NLP to production is best addressed by adopting MLOps, which is a fancy way of saying Machine Learning on top of DevOps.

MLOps introduces automation between ML notebooks with DevOps pipelines. An MLOps platform handles the tough task of integrating multiple tools by solving interdependencies at various stages of development AND deployment. For example Github, Github Actions, Jenkins, monitoring tools, etc. These can be tools that data scientists are often wholly unfamiliar with. So, how can MLOps data scientists solve this operational challenge? There are a variety of options, but if the following matter to the organization:

• Portability: Write once, reproduce and run everywhere
• Microservices: Workflows often need to interact with multiple services
• Scaling: Scaling down quickly can be just as important as scaling up

…then Kubeflow running on top of Kubernetes is the obvious choice… Kubeflow is an open-source platform for scalable machine learning model training and serving.

Kubeflow offers a “Runs”’ feature that helps connect a Jupyter Notebook containing Natural Language Models pipeline scripts and model creation scripts with Kubeflow Pipelines. Once connected, the workflow is converted into YAML format and is ready for deployment. It now becomes much easier for data scientists to quickly deploy NLP models. When Kubeflow is combined with the Kale add-on, data scientists can easily set up multiple hyper-parameters so that the model with the greatest accuracy (given the requirements) can be the one that is ultimately served. Also, Kubeflow offers built-in utilities to monitor the output, further reducing the burden on data scientists.

Below is an example of ML workflow being run inside Kubeflow.

Conclusion

The ultimate goal of any organisation that uses Natural Language Processing (NLP) is for their language models to successfully run in production and generate value for the business. However, many steps must be completed before a model can be deployed in production, including data loading, verification, splitting, processing, feature engineering, model training and verification, hyperparameter tuning, and model serving. Furthermore, even though your data inputs can drift over time, NLP models may necessitate more observation than traditional applications. Manually rebuilding models and data sets takes time and are prone to error. 

Kubeflow can easily solve issues and challenges like:

• Deploying and managing a large-scale NLP system
• Experimentation with training an NLP model 
• End to end hybrid and multi-cloud NLP workloads
• Tuning the model hyperparameters during training 
• Continuous integration and deployment (CI/CD) for NLP pipelines