6 Algorithmic Breakthroughs Of 2020

With the increase in efficient use-cases, the interest in the field of artificial intelligence is gaining a hot seat among researchers and organisations. Despite the pandemic, 2020 has witnessed several interesting developments in the domain of AI and machine learning. With a number of developments going around along with the constant digitisation, even this year, emerging technologies like AI have proved to be more intelligent as well as complex than humans.

Here is a list of the algorithmic breakthroughs that were made in 2020.

Note: This list is in no particular order.

ReBeL

Developed by the researchers at Facebook AI Research (FAIR), Recursive Belief-based Learning or ReBeL is a general RL+Search algorithm that can work in all two-player zero-sum games, including imperfect-information games. The algorithm is built on the RL+Search algorithms like AlphaZero, and it makes decisions by factoring in the probability distribution of different beliefs of each player about the current state of the game. 

According to the official blog post, the experimental results showed that ReBeL is effective in large-scale two-player zero-sum imperfect-information games like Liar’s Dice and poker, where the algorithm even managed to defeat a top human professional in the benchmark game of heads-up no-limit Texas Hold ’em.  

Know more here.

Efficient Non-Convex Reformulations

The algorithm Efficient Non-Convex Reformulations is introduced by Alphabet’s DeepMind. It is a verification algorithm and a novel non-convex reformulation of convex relaxations of neural network verification. The method automatically generates a sequence of primal and dual feasible solutions to the original convex problem, making optimality certification easy. 

According to its developers, this new scalable algorithm leads to verifying properties of neural networks and solves certain kinds of structured regression problems. It can have an impact in terms of better methods to evaluate the reliability and trustworthiness of state of the art deep learning systems, thereby catching any unseen failure modes and preventing undesirable consequences of deep learning models.

Know more here.

Memory-Efficient First-Order Semidefinite Programming

Memory-Efficient First-Order Semidefinite Programming is a first-order dual SDP algorithm that requires memory only linear in the total number of network activations and only requires a fixed number of forward/backwards passes through the network per iteration.

With this algorithm, the developers at DeepMind tried to exploit the well-known reformulations of SDPs as eigenvalue optimisation problems. According to the experimental results, the authors claimed that this approach could lead to scalable and tight verification of networks trained without the need for special regularisers to promote verifiability.

Know more here.

Advantage Weighted Actor-Critic (AWAC)

Advantage Weighted Actor-Critic (AWAC) is a machine learning algorithm that learns from offline data and fine-tunes in order to reach expert-level performance after collecting a limited amount of interaction data. It is able to quickly learn successful policies on difficult tasks with high action dimension and sparse binary rewards, significantly better than prior methods for off-policy and offline reinforcement learning. 

According to its developers, the algorithm solves the rotation of pen tasks in 120K timesteps, the equivalent of just 20 minutes of online interaction. Also, the algorithm can utilise different types of prior data, such as demonstrations, suboptimal data, and random exploration data.

Know more here.

RigL Algorithm

A few months ago, Google introduced a new algorithm for training sparse neural networks, known as RigL algorithm. The algorithm identifies which neurons should be active during training, which helps the optimisation process to utilise the most relevant connections and results in better sparse solutions. RigL has the capability to improve the accuracy of sparse models intended for deployment as well as the accuracy of large sparse models that can only be trained for a limited number of iterations. 

Know more here.

Behaviour-Regularised Model-ENsemble (BREMEN)

Behaviour-Regularised Model-ENsemble or BREMEN is a model-based algorithm developed by researchers from Google Research along with the University of Tokyo. BREMEN algorithm has the ability to optimise an effective policy offline using much lesser data. BREMEN learns an ensemble of dynamics models in conjunction with a policy using imaginary rollouts while implicitly regularising the learned policy via appropriate parameter initialisation and conservative trust-region learning updates. 

The researchers found out that BREMEN can not only achieve performance competitive with state-of-the-art when using standard dataset sizes but also learn with 10-20 times smaller datasets, which previous methods are unable to attain.
Know more here.