As Elon Musk had promised earlier, Twitter has decided to make its algorithm code that it uses for selecting tweets to display on users’ timelines publicly available on GitHub and provided an explanation in a blog post. The post outlines the factors considered by the algorithm in selecting tweets for the “For You” timeline and how they are ranked and filtered.
Click here to check out the code.
Musk revealed much information about the release on a Twitter Spaces session earlier. He pointed out that the release might look quite embarrassing as there are a lot of mistakes in the code. Interestingly, to avoid any risk, the open-source code does not include Twitter’s ad recommendations or the social media’s training data.
The algorithm consists of three stages, as per Twitter’s explanation. The algorithm collects the most noteworthy tweets from various sources, ranks them using a machine learning model, and then removes any tweets from blocked accounts, previously seen tweets, or NSFW content before displaying them on a user’s timeline.
Manu Joseph, creator of PyTorch Tabular, said in a post that It’s essential to note that these algorithm system are a complex collaboration, that require many different components. A basic ranking algorithm or collaborative filtering model isn’t adequate on its own.
Interestingly, the blog shows that the pipeline above runs approximately 5 billion times per day and completes in under 1.5 seconds on average. Moreover, a single pipeline execution requires 220 seconds of CPU time, nearly 150x the latency you perceive on the app.
Twitter Strides
The release of the code of recommendation algorithm’s source code, comes after a series of controversies in recent months. Platformer reported that in February, Elon Musk requested that Twitter’s engineers modify the algorithm to increase the visibility of his tweets.
However, Twitter later reversed this alteration. In November, Twitter started displaying more tweets from non-followed accounts, a move that had been tested before Musk’s involvement but was ultimately scrapped after user pushback.
In the release Musk talks about a lot of privacy and security issues for not including a lot of code for the website. On the flip side, this comes in the backlight of Twitter recently firing its ethical AI and safety staff. But even then, Musk has ensured that the company has taken all the steps to ensure the reliability and security of the code.
Twitter is also facing competition in the open-source community. Mastodon, the decentralised social network platform has been gaining traction among several Twitter users. Jack Dorsey, Bluesky is also attracting a lot of users, which is also looking for open-source protocols.
