Machine learning is one term that has created an immense amount of buzz in the technology industry. With its enormous potential in healthcare, medical diagnosis as well as solving complex business problems, machine learning has revolutionised many aspects of human lives.
However, as the technology is evolving and becoming more complex, it comes up with exciting subfields and terminologies associated with it. In this article, we are going to decode some of the most used jargons in machine learning.
Also Read: 25 AI Terminologies & Jargons You Must Assimilate To Sound Like A Pro
Autoregression
Autoregression is a phenomenon in machine learning, where an autoregressive model learns from a series of times steps, aka a time series model that uses information from previous timed steps as input to a regression equation in order to predict the value. With autoregression, one can predict accurate forecasts on a range of time series problems.
It works based on determining the correlation of previous time steps, also known as the variables, among each other, which in turn helps in predicting the output. If both the variables changed in the same direction, there could be a positive correlation; however, if both turned to a different direction, then it will be termed as negative — either of the ways the relationship between the result and the input can be easily determined. With a higher correlation, the more chances of predicting the outcome from the past information.
To understand better read: Python Library For Time Series Analysis And Prediction.
Backpropagation
In machine learning, backpropagation is also known as “backward propagation of errors,” and is an algorithm used for training artificial neural networks for supervised learning. It works by determining the minimum value of the error at the output and then propagating it back into the neural network. Backpropagation is a critical process of neural net training, where it is leveraged for fine-tuning the weights based on the error rates. Proper modification of weights will help in minimising the errors, making the model reliable.
Not only this method is fast and easy to program, but also needs no parameters to tune and no prior knowledge about the network. Static backpropagation and recurrent backpropagation are the two types of backpropagation networks. Although it comes with many benefits, the only drawback is the sensitivity of the method for noisy data.
You can also read what Geoff Hinton thinks of Backpropagation.
Few-shot Learning/One-Shot Learning
Few-shot learning is the type of model training where a very small set of training data is used instead of an extensive one. It carries a suitable object categorisation model work without several training examples. The best way for this is by learning the common representation of various tasks and train them on task-specific classifiers. It is best used when acquiring enough data to optimise the model isn’t possible, thus, few-shot learning can be beneficial for such scenarios in order to identify the patterns in the data and predict the outcome.
Whereas, one-shot learning is an approach in machine learning where the model has been trained only a single instance, usually used for object classification. One of the renowned examples of one-shot learning is the facial recognition system that people deploys. Typically to train deep learning systems, one would require several input data with pictures of human faces for them to finally identify that face in the crowd. However, it is never determined that train the model with images; it might not naturally react when it identifies a new face. Thus one-shot learning comes handy, where the conventional neural network is being trained to learn a distance function between images instead of classifying them.
To understand better, read this: Will One-Shot Learning Using Hypercubes Outrank Traditional Neural Nets?
Also Read: Road To Machine Learning Mastery
Hyperparameter
In machine learning, model parameters are known as the properties of training data which can learn independently during training by machine learning models. Some of the critical parameters are weights and biases. With that being said, a hyperparameter is a type of parameter that dictated the entire training process of the model. Its value is decided before training the model and is basically used to control the learning process. On the other hand, usually, the values of different parameters are derived with model training.
The number of hidden units and learning rate are some of the key model hyperparameters used. With direct control of the training process, hyperparameters are critical in measuring the performance of the model. Thus, choosing appropriate hyperparameters can provide easier management of extensive experiments. Hyperparameters can be of two categories — optimiser and model specific.
For a better understanding of hyperparameter optimisation tools, read this.
Recommendation Engine
Recommendation Engine is basically used for recommending customers their favourite products on an online platform. It is a data filtering tool that uses algorithms to recommend the most relevant and preferred items for a particular user. To facilitate this, firstly, it identifies the past behaviour of the customer and leveraging that information, it suggests products and items for the customers. However, for customers with no prior buying pattern, the engine proposes products that are the best selling for the site.
Apart from recommending products, the recommendation engine also segments the customers according to their preferences and manages them accordingly for future buys. In order to build a successful recommendation system, one has to accumulate data about the products, demographics, as well as the customer. The data is then filtered to extract relevant information to make the required suggestions.
To understand how Google made it super easy to build a recommendation engine, read this.
Tokenization
Tokenization is the process of transforming data into tokens, explicitly used for NLP. For instance, if the data is an account number; then this process will turn that account number into a string of characters also called tokens. These tokens are used as a reference for the data but cannot be used to guess those values. Being the building blocks of natural language, tokens have been the most leveraged way of processing data.
Some of the key types of tokenization include — word tokenization, character tokenization and subword tokenization. Tokenization can be done via many approaches, whether it be white space tokenization or dictionary-based tokenization. In white space tokenization, the input sentence is broken apart every time a white-space is encountered, whereas, the dictionary-based tokenization, determines tokens from the sentences that are already in the dictionary.
For better understanding, read: Hands-on Guide to StanfordNLP
Optimizer
As the name suggests, optimizers are the type of algorithms and techniques used to optimise the neural networks during its training process; thus, the method is known as ‘optimization.’ These algorithms are leveraged to modify the attributes of the neural network, such as the learning pace or the weights in order to minimise the loss function as well as determine accurate results. In layman terms, the optimisers transform the model in a perfect form by working around its weights to determine precise results.
One of the most popular optimisation algorithms used is the ‘Gradient Descent,’ which works on linear regression and classification algorithms to calculate the attributes of the neural networks. Other types of optimisers are — Momentum, Adagrad, Root Mean Square Propagation (RMSProp), Ada Delta, Nesterov and Adam, to name a few.
To know more about optimisers, read — Guide To Optimizers For Machine Learning.
Convergence
Convergence is a method or technique where an iterative algorithm aka the initial method to predict the outcome, converges when the result gets closer to a particular value. To be exact, not depending on the error range, if convergence continued long enough, the function would eventually remain within that mentioned error range around the final value.
In layman terms, when the data is processed over several times, the model converges to represent latent variables or the errors throughout the weights of the neutrons in the network. It is also believed that inappropriate convergence would lead to longer training time and massive data for the machine learning algorithm.
Learning Rate Annealing
The process of training neural networks involves several hyperparameters, and one of the key, among those, is the learning rate for gradient descent. This hyperparameter determines the magnitude of the weights in order to reduce the losses. The speed of the learning rate is equivalent to the process of training the neural networks. However, if the learning rate is higher than the desired, It can create undesirable divergences in the loss function. Now, to adjust the learning rate, one of the fundamental techniques is learning rate annealing, which starts by recommending a high learning rate and then gradually slowing it down during the training process.
This again comes in many forms, and the most popular way of annealing the learning rate is by ‘step decay,’ which reduces the learning rate to a certain extent after trying out some training methods. In fact, it is critical to define the learning rate schedule, where it can get updated during the training process and can be done with annealing.
To understand better read: Top Optimisation Methods In Machine Learning.
Batch Normalization
Similar to other techniques, batch normalisation is a method of making the neural network faster and more stable by re-centring and re-scaling. To explain the process, this technique normalises the input layer by modifying and scaling the activations and allows each of the layers to learn independently, minimising or normalising the hidden layers.
With batch normalisation, one can also increase the learning rate, as the technique ensures the activation to be in the right value — neither high nor low. With fewer regularisation effects, it can also reduce the overfitting of the model. Further, leveraging the process of batch normalisation can reduce the change in the distribution of input variables, thus speeding up the training process.
