A Guide to Stochastic Process and Its Applications in Machine Learning

Many physical and engineering systems use stochastic processes as key tools for modelling and reasoning.  A stochastic process is a probability model describing a collection of time-ordered random variables that represent the possible sample paths. It is widely used as a mathematical model of systems and phenomena that appear to vary in a random manner. As a classic technique from statistics, stochastic processes are widely used in a variety of areas including bioinformatics, neuroscience, image processing, financial markets, etc. In this post, we will discuss the stochastic process in detail and will try to understand how it is related to machine learning and what are its major application areas. The major points to be discussed in this article are outlined below.

Table of Contents

1. Stochastics in General
2. Stochastic Process
3. Examples of Stochastic Processes
4. Comparing Stochastic Systems with Other Systems
5. Stochastic Process in Machine Learning
6. Application of Stochastic Process

Let’s start by knowing the general meaning of stochastic.

Stochastic in General  

Stochasticity is the property of being well described by a random probability distribution. Although stochasticity and randomness are distinct in that the former refers to a modelling method and the latter to phenomena, the terms are frequently used interchangeably. Furthermore, the formal concept of a stochastic process is also referred to as a random process in probability theory.

Stochasticity is employed in a variety of domains, including biology, chemistry, ecology, neuroscience, and physics, as well as image processing, signal processing, information theory, computer science, cryptography, and telecommunications. It’s also employed in health, linguistics, music, media, colour theory, botany, manufacturing, and geomorphology, all of which are affected by seemingly random movements in financial markets. In social science, stochastic modelling is utilized.

---

---

Stochastic Process

Although the definition of a stochastic process varies, it is typically characterized as a collection of random variables indexed by some set. Without the index set being clearly described, the phrases random process and stochastic process are considered synonyms and are used interchangeably. The phrases “collection” and “family” are used interchangeably, whereas “parameter set” or “parameter space” are occasionally used instead of “index set.”

Some theoretically defined stochastic processes include random walks, martingales, Markov processes, Lévy processes, Gaussian processes, random fields, renewal processes, and branching processes. Probability, calculus, linear algebra, set theory, and topology, as well as real analysis, measure theory, Fourier analysis, and functional analysis, are all used in the study of stochastic processes.

Example of Stochastic Process

Poissons Process

The Poisson process is a stochastic process with several definitions and applications. It’s a counting process, which is a stochastic process in which a random number of points or occurrences are displayed over time. A time-dependent Poisson random variable is defined as the number of points in a process that falls between zero and a certain time. Non-negative numbers make up the index set of this process, but natural numbers make up the state space. Because it can be conceived of as a counting operation, this procedure is often referred to as the Poisson counting process.

Bernoulli Process

One of the most basic stochastic processes is the Bernoulli process. It’s a set of independent and identically distributed (iid) random variables, each with a probability of one or zero, for example, one with probability p and zero with probability 1-p. This method is similar to repeatedly flipping a coin, with the chance of getting a head being p and the value being one, and the probability of receiving a tail being zero. A Bernoulli process, in other words, is a set of iid Bernoulli random variables, with each coin flip representing a Bernoulli trial.

Random Walk

The simple random walk is a typical example of a random walk. It is a stochastic process in discrete time with integers as the state space and is based on a Bernoulli process, with each Bernoulli variable taking either a positive or a negative value. In other words, the simple random walk occurs on integers, and its value grows by one with probability p or lowers by one with probability 1-p, hence the index set of this random walk is natural numbers, but its state space is integers. If p=0.5, this random walk is referred to as a symmetric random walk.

Comparing Stochastic Systems with Other Systems

Let’s compare stochastic systems to other similar terms that are occasionally used as synonyms for stochastic to gain a better grasp of it. Stochastic is synonymous with random and probabilistic, although non-deterministic is distinct from stochastic.

Stochastic Vs Probabilistic 

The terms stochastic and probabilistic are frequently interchanged. Probabilistic is most likely the broader term. Stochastic is dependent on a previous occurrence, such as fluctuations in stock price based on the previous day’s price, but probabilistic is independent of other observations, such as winning lottery numbers, which are supposed to be independent of one another.

Stochastic Vs Non-Deterministic

Deterministic refers to a variable or process that can predict the result of an occurrence based on the current situation. In simple terms, we can state that nothing in a deterministic model is random. Non-deterministic, on the other hand, is a variable or process in which the same input might result in different results.

Because the outcome is unpredictable, stochasticity is often used interchangeably with non-deterministic methods. In the way that we may undertake analysis using probability tools like anticipated result and variance, stochasticity is slightly different from non-deterministic. As a result, defining a variable as stochastic rather than non-deterministic is a stronger claim.

Stochastic Vs Random

In most cases, stochastic is used interchangeably with random. Random refers to unpredictability, and in the ideal scenario, all outcomes are equally likely, implying that there is no reliance on the other observation, such as tossing a fair coin, whereas stochastic refers to the probabilistic nature of the variable that is randomly chosen.

Stochastic Processes in Machine Learning

Stochasticity is used to explain several machine learning methods and models. This is due to the fact that many optimizations and learning algorithms must function in stochastic domains, and some algorithms rely on randomness or probabilistic decisions. Let’s look at the source of uncertainty and the nature of stochastic algorithms in machine learning in more detail.

How it is Identified in Machine Learning

Domains involving uncertainty are known as stochastics. Statistical noise or random errors can cause uncertainty in a target or objective function. It could also be due to the fact that the data used to fit a model is a sample of a larger population. Finally, the models adopted are rarely able to capture all elements of the domain, and must instead generalize to unknown scenarios, resulting in a loss of fidelity.

Optimizing the Stochastic

Optimization approaches that create and employ random variables are known as stochastic optimization (SO). Random variables exist in the formulation of the optimization problem itself for stochastic issues, which incorporates random objective functions or random constraints. Random iterate methods are also included in stochastic optimization approaches. Some stochastic optimization approaches combine both definitions of stochastic optimization by using random iterates to address stochastic issues.

The following are some instances of stochastic optimization algorithms:

• Particle Swarm Optimization
• Simulated Annealing 
• Genetic Algorithm

Stochastic Learning Algorithm

The two most prevalent and widely used algorithms in machine learning are stochastic gradient descent and stochastic gradient boosting.

Stochastic gradient descent (SGD) is a variant of the gradient descent technique that computes the error and updates the model for each example in the training dataset. Because the model is updated for each training example, stochastic gradient descent is frequently referred to as an online machine learning algorithm.

The stochastic gradient boosting algorithm is a collection of decision tree techniques. The stochastic aspect refers to the random subset of rows drawn from the training dataset that are utilized to build trees, specifically the split points of trees.

Application of Stochastic Process

Below are some general and popular applications which involve the stochastic processes:-

• Stochastic models are used in financial markets to reflect the seemingly random behaviour of assets such as stocks, commodities, relative currency values (i.e., the price of one currency relative to another, such as the price of the US Dollar relative to the price of the Euro), and interest rates. 

• Manufacturing procedures are thought to be stochastic. This assumption holds true for both batch and continuous manufacturing processes. A process control chart depicts a particular process control parameter across time and is used to record testing and monitoring of the process.

• The marketing and shifting movement of audience tastes and preferences, as well as the solicitation and scientific appeal of the certain film and television debuts (i.e., opening weekends, word-of-mouth, top-of-mind knowledge among surveyed groups, star name recognition, and other elements of social media outreach and advertising), are all influenced in part by stochastic modelling.

• Stanislaw Ulam and Nicholas Metropolis popularized the Monte Carlo approach, which is a stochastic method. The use of randomness and the repetitive nature of the procedure is reminiscent of casino activities. Simulation and statistical sampling methods were typically used to test a previously understood deterministic problem, rather than the other way around. Though historical examples of an “inverted” technique exist, they were not regarded as a generic strategy until the Monte Carlo method gained popularity.

Conclusion 

In this post, we understood the stochastic process with different concepts and application areas. We went through the definition of stochastic, how it differs from related terms like random, probabilistic, and nondeterministic, and what stochastic means in machine learning in this post. In addition, we have some fundamental and common examples of stochastic processes that can be seen in generic terms. 

References 

• Stochastic Process
• Stochastic Optimization
• What Does Stochastic Mean in Machine Learning?