The relationships within data can be represented with nodes. From recommending mutual friends on social networking sites or interactions of proteins, graph-based reasoning can be applied to uncover hidden interactions or how one node is related to previously abandoned ideas. 

The use of graph-based representations for web applications has been around for a while now.

Large scale knowledge graphs are usually known for their ability to support NLP applications like semantic search or dialogue generation. Whereas, companies like Pinterest already have opted for a graph-based system (Pixie) for real-time high-performance tasks like image recommendation.

How Topological-Based ML Gained Traction

In neural networks, vector representations play a crucial role in connecting input layers to other layers. But graphs are by nature combinatorial structures made of nodes and edges while neural networks favour continuous vector representations.

To address these challenges and to make the most out of graph-based systems for neural networks, different approaches have been introduced — one of them being the learning of multiple representations of the nodes in a graph. Here each representation encodes the role of the node and allows for an effective visual analysis of the learned community structure. Communities are units of a neural network that have similar connection patterns with the adjacent layers within the network. 

In other words, this is an attempt to find the relations between data points and making the neural network to predict better.

Automating Hyperparameter Tuning With Graphs

The above picture is an illustration of a typical graph with highly overlapping communities on the top right and a persona graph on bottom left that disentangles the overlapping communities of the original graph.

To improve the participation between these communities a new method called Splitter has been introduced. In this method, a graph G is taken and a new graph P is created called the persona graph. 

Each node in G is represented by a series of replicas called the persona node And, these nodes are analysed to discover local communities to which the node belongs to.

This method is significant in real-world settings where there is a lot of overlap between the communities. For example, if a recommender system has to be developed based on the likes of a person and what their friends like then there will be overlap. Now extrapolate this scenario from individual to a nation where the data has to be explored for surveys on medical diagnosis or other such delicate scenarios where the presence of outliers and their connection to a data point is highly probable. 

Graph embedding methods like DeepWalk explore the context around a given node. This is a hyperparameter tuning technique through an exploration of neighbouring nodes. And, to identify which part of the graph requires more attention. To optimise and automate the hyperparameters, Google introduced Watch Your Step, an approach that formulates a model for the performance of embedding methods. In short, making the graph to concentrate on direct significant neighbours. Here the “Auto” portion corresponds to learning the graph hyperparameters by backpropagation.

Future of Graph Adoption

The success of graph-based embeddings for representations within a neural network have great implications on the development of automated machine learning. The rise of this new wave of topological based machine learning approaches is a direct indication of the researcher’s effort to alleviate the burden of optimising hyperparameters. This is done through various mathematical techniques like the use of the connection between hyperparameters in the embeddings and graph theoretic matrix formulations. This domain also poses challenges like the use of multiple-embedding methods for classification, which the researchers believe will get significant attention in future ML endeavours.

Know more about graph based hyperparameter tuning here