TikTok Parent ByteDance Reveals its SOTA Recommendation Engine

Earlier this week, we released a story on how TikTok has revolutionised the short-video industry through its recommendation system. In just five years, the platform acquired about 1.2 billion monthly active users (as per Q4 2021) and is estimated to reach 1.8 billion users by the end of year. 

Today, tech giant ByteDance revealed the main structure of ‘Monolith’, TikTok’s recommendation system’s algorithm. TikTok has undoubtedly taken over the internet by basically reading your mind to get personalised content. 

Inside TikTok Recommendation Engine 

TikTok is undoubtedly one of the fastest growing social media services and several researchers have credited the app’s success to produce their recommender system algorithm. Now, the secret is out. Researchers at Bytedance in a recent paper titled, “Monolith: Real Time Recommendation System With Collisionless Embedding Table“, showcased the model that is used for online training. 

Building a real-time and scalable recommendation system is vital for various businesses driven by customer feedback which is time-sensitive, such as online ads and short-videos ranking.

The paper cites that despite the adoption of deep learning frameworks like PyTorch or TensorFlow, such frameworks have fallen short of demands in business recommendation scenarios for several reasons. This includes tweaking systems based on dense computations and static parameters for recommendation with dynamic features, which is detrimental to model quality. 

The researchers believe that these frameworks are designed/built with batch-training stage and serving stage completely separated, which prevents the model from interacting with customer feedback in real time.

The research reexamines traditional approaches, mainly exploring different design choices tailored for online training. 

However, the design has been driven by observations of production environment and application workloads—thereby reflecting a departure from other recommendation systems. 

“Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provided a production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product,” asserts the paper. 

Source: Monolith: Real Time Recommendation System With Collisionless Embedding Table

Read: Nothing like TikTok, Kudos for Trying

In 2019, the Indian government banned TikTok. This led to the rise of homegrown short video platforms such as Chingari, Mitron, Josh, Taka Tak, Moj, Roposo with several other platforms coming into existence later to fill the gap. 

Every social media company is trying to become like TikTok, but remain unsuccessful so far. Major tech companies with their social media platforms have been constantly trying to overhaul with features similar to TikTok. Meta, for the first time in its history of 18 years, reported a loss in the count of its daily users in 2022. Moreover,  in the company’s Q1 2022 earnings call, Meta’s CEO Mark Zuckerberg called blame towards TikTok for user stagnation.  

Learnings so far 

ByteDance researchers said that they have reviewed several high priority challenges for industrial-level recommendation systems and presented their system in production, Monolith, to address them and achieve best performance compared to existing solutions. 

Further, they believe that collisionless embedding tables are essential for model quality and showed that their implementation of Cuckoo HashMap-based embedding tables is both memory efficient and helpful for improving online serving metrics. 

In addition to this, they also proved that realtime serving is crucial in recommendation systems, and that parameter synchronisation intervals should be as short as possible for an ultimate model performance. “Our solution for online real time serving in Monolith has a delicately designed parameter synchronisation and fault tolerance mechanism,” said the researchers. 

For instance, in their parameter synchronisation algorithm, they showed that consistency of version across different parts of parameters could be traded-off for reducing network bandwidth consumption. In the fault tolerance design, the researchers demonstrated that their strategy of trading-off PS reliability for realtime-ness is a robust solution. In other words, its Monolith’s success in providing a general solution for production scale recommendation systems.