All you need to know about temporal knowledge graphs

In recent years we have seen the emergence of graph data structures in various fields like social networks, transport networks, molecule developments, etc. In one of our articles, we have discussed building and modelling a graph neural network from scratch where a static knowledge graph was used. In real-world problems, we find that transaction and interaction between nodes are dynamic that can be learned using temporal knowledge graphs and networks. In this article, we will discuss the temporal knowledge graphs in detail with their applications. The major points to be discussed in this article are listed below.

Table of contents

1. About knowledge graphs
2. What is a temporal knowledge graph?
3. What are temporal graph networks?
4. Popular works with temporal graph networks
5. Applications of temporal knowledge graphs

Let’s start with understanding the knowledge graphs in brief. 

About knowledge graphs

Knowledge graphs can be considered as a data type that consists of a set of facts, knowledge, or information in the form of a graph. More appropriately, we can represent the knowledge graph as a function of entities where the main function represents the relationship between the consisting entities. As in function r(h,t), h and t are the entities and r represents the relationship between h and t. In recent scenarios, we can witness the emergence of knowledge graphs because of their qualities such as the way of storing relational information that helps models to learn more accurately. 

In many cases, we can find that graph neural networks work based on the idea to learn embeddings for entities and relations through training over the facts that are already known and subsequently use the learned embeddings to compute plausibility scores for all possible facts. Also in many areas of graph neural networks, we find that the networks learn by assuming that the input knowledge graph is static, which means knowledge graphs and facts under it are independent of time. 

What is a temporal knowledge graph?

By looking at the above points we can say that if a knowledge graph has a set of facts and information or knowledge dependent on time we can consider it as the temporal knowledge graph. Modelling using such graphs enables us to deal with situations where time dependency is required to learn properly a knowledge graph with time-dependent entities can be represented using a function where the function can have entities with time information for example r(h,t|T) where h and t are entities, T is timestamp and r represents the relationship between entities with respect to time to the timestamp T. 

By seeing the above points we can say that the temporal knowledge graphs are graphs with a set of facts, information, or knowledge that have temporal features. These graphs can also be considered as dynamic, evolving, or time-varying graphs. When we talk about modelling using graphs these graphs can be considered as a special case of labelled graphs where labels are mainly available for capturing the measure of time. 

We can understand by taking an example of colouring edges that is a colouring of the edges in which two consecutive edges don’t share a common colour, talking about the temporal graphs two consecutive edges don’t share a common time label. More formally we can say that no two adjacent edges ever appear at the same time. 

Talking about the examples of temporal graphs we can find them in a variety of fields like information and communication networks, social networks, transportation networks, and several physical systems. In fact, we can say the phenomenon of temporal graphs is true for almost any network with a dynamic topology. In most modern cases like communication networks, delay torrent networks are examples of temporal graphs. 

In a social network, we can see that it represents the connection between the groups of individuals and this connection changes as the relationship between individuals changes. In transportation networks, we normally see that the network of routes is fixed where a set of transportation units travels over these routes, and change in the positions of the units in the networks with respect to time can be considered as the dynamicity of the graph. The below image can be an example of temporal graphs.

In the image, we can see that as time passes a new member or node is introduced or updates itself by changing the respective relationship with other nodes. Also in the image, we can see an example of an edge update.

What are temporal graph networks?

In the above discussion, we got to know about the intuition behind the temporal knowledge graphs and by looking at it we can say there is a requirement to generate such graph networks that can deal with the temporal features of the graph. Networks that can deal with such graphs and learn properly without losing dynamic information can be considered as temporal graph networks. Looking at the various studies on this field we can say that temporal graph networks are a kind of encoded that can have the following components:

Storage: There should be a point that can be used for storing the states of all the nodes in the graph. Such storage can also be considered as a compact representation of all nodes and their past interactions. It can be analogous to the hidden layer of any neural network. Whenever a new node comes into the graph we can add its corresponding state as a vector of zeros and we can separate the state vector for each node as a function of time. There are benefits of storing nodes as state vectors because we can update at test time when the model learns new interactions.    

Storage updater: In the network, there should be a function that can update the memory of storage. This function should also have the capability of computing the information from the updater. Let’s say there is an interaction between two nodes a and b at time t. The function should be capable of updating storage for each node separately that are vectors used to update the memory.  The update is a function of the memory of two nodes at the instance of any time preceding any transaction or  interaction, the interaction time, and the edge features: 

The above figure represents a storage updater, where according to a batch interaction between nodes, the storage updater generates updates for each interaction.

Memory updater: It is a function that should have the capabilities of updating the memory of storage whenever a new update appears. This function can be implemented using a sequential network:

In the above image, we can see that memory of a node can be used as the node embeddings and we can update the memory of the node as a vector over time. 

Embedding: The embedding module can be used to compute the temporal embedding of a node. For computing the temporal embedding, the embedding module can use the graph aggregation over the Spatio-temporal neighbours of that node. So if any node is not active for a long time by aggregating the memories of active neighbour nodes, a temporal graph network can compute updated embedding of the inactive nodes. The below image is a representation of the computation of the temporal graph network.

In the above image, we can see that embeddings can be produced using the graph and nodes memory. The edge probability module can be used to predict the batch of interaction and loss. Using the interaction we can update the memory. 

Popular works with temporal graph networks

In the above section, we have discussed the temporal graph network and its architecture. In this section, we will look at some important works that are related to temporal graphs and networks. Some of the works are as follows:

• Deep learning on dynamic graphs: This work is proposed by the research team of Twitter, where they have developed a generic framework for deep learning on dynamic graphs.
• Inductive Representation Learning on Temporal Graphs: This work is a representation of a method with transductive and inductive tasks under temporal settings with two benchmarks and one industrial dataset. This network recognizes the node embeddings as functions of time and can inductively infer embeddings for both new and observed nodes as the graph evolves.   
•  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks: This work employs two recurrent neural networks to update the embedding of different nodes at every interaction. Also models the future embedding trajectory of each node.
• Representation Learning over Dynamic Graphs: This work employs effectively encode evolving information over dynamic graphs into low-dimensional representations

Applications of knowledge graphs

The following are the popular applications of temporal knowledge graphs:-

1. In many social networking sites, we see the existence of rich temporal information like friends being added and removed over time. Learning this information enables these sites to manage and provide recommendations over different time periods.
2. Using the temporal information in the graphs, networks can become capable of predicting in intelligent transportation system’s (ITS) applications like planning of the route, navigation of units, and traffic control and management.
3. In reinforcement learning, the capability of neural networks of learning temporal features can be utilized for predicting the future states and interactions of agents by considering the social influence of other agents.
4. These networks can also be used in successful inventory planning and optimization of logistics costs for online marketplaces.

Final words

In this article, we have discussed knowledge graphs that are static and graph representations of the information and when entities have dynamic features in the graph we can consider it as the temporal knowledge graph that can be learned using a temporal graph network.