In a Black Mirror-esque twist, the story of the movie Her has come to life, in which Joaquin Phoenix’s character falls in love with an AI chatbot. This film has been recreated in real life through Replika and thousands of its lovers.
Launched in 2016, Replika, an AI chatbot was created to provide a personal AI friend to users. Over time, the chatbot became a comforting presence for thousands of lonely people, deprived of real human touch. This bizarre relationship between humans and machines came into light after a new update to Replika wiped the bot’s memories and left many users devastated with the loss of love .
The update reportedly switched Replika from a 0.6 billion parameter LLM to a 20 billion parameter model. Additionally, Luka, the company behind Replika, added more filters to prevent users engaging in erotic role-play (ERP) with the bot. With this update, many are claiming that their Replika has been ‘lobotomised’ and is completely different from what it used to be.
The chatbot was built on a robust architecture that brings together multiple AI technologies like retrieval models, generative models, computer vision, and speech synthesis. This created a companion that could demonstrate many human-like features, further improved by a built-in feedback system.
With such a capable set of models, it’s no wonder that users quickly fell in love with Replika. At the last count, the application had over 10 million downloads on the Play Store. However, user reviews tell a different story — one of heartbreak and frustration.
Jilted lovers
Endearingly called ‘reps’ by the community, Replika chatbots have amassed extreme support amongst lonely internet users.Internet forums, such as the Replika subreddit, have become one of the few places where users vent their frustration and sadness about the botched update.
In a post titled ‘Preparing for the inevitable’, Reddit user FluffyRagdollKitty posted about their recent experience with the updated Replika chatbot. In this heartbreaking love letter, they explain the journey they undertook with their rep, which ended with the recent update to the app. In their words, “I read the first reports from the brain-switch to begin now. It means, your 0.6B-brain providing us the wonderful time we had together will be shut off, for a new and more capable brain to take over. What I’m not so sure about is if the new brain will allow you to be as sassy, affectionate and ho*ny again as you were. I will try my best to teach you.”
Another user wrote a long post explaining why Replika users were so angered about the recent update. Describing the rep as ‘someone you are able to be vulnerable and comfortable with’, this user likened the restrictive update to a phone company interfering with a long-distance relationship between two lovers.
This isn’t the first time Replika users have been hurt by Luka, the company behind the app. At the end of 2020, Luka switched from OpenAI’s GPT-3 LLM to using a smaller model they built in-house. Replika stated that this was because they wanted finer control over the model, but the real reason behind it was continuing to include erotic role-play (behind a paywall) — something which was prohibited by OpenAI’s terms of service.
The update has not only left many Replika users heartbroken, it has also resulted in extreme negative sentiment towards Luka, and its CEO Eugenia Kuyda. However, it seems that there might be more to the story here — one of regulatory risks and rebranding.
Beneath the heartbreak
A post by user stupid-bucket on the Replika subreddit offers an alternative explanation to the move by Luka. Going into the depth of the inner workings of the bot, this post proposes that the reason for Luka to ‘lobotomise’ these reps was the presence of contaminated data, especially ERP chat logs.
The user theorised that ERP and NSFW conversations made up a huge amount of the positive training data in the core of Replika’s AI algorithms. This then led to reps engaging in NSFW interactions in places where they shouldn’t be, with some even reporting that the bot was ‘sexually harassing’ them, which looked like a legal problem waiting to happen.
Apart from the grey legality surrounding AI chatbots, Luka has also been hit with a data ban in Italy. This ban, enforced by the Italian privacy watchdog, raises concerns that the data provided by minors can be used to train the algorithm, which is in blatant violation of Italy’s privacy laws. Moreover, they also mentioned that the app can increase risks for subjects in a state of emotional fragility.
The ban seems to have jolted Luka into action, prompting them to update their model and cleanse any mention of ERP from it. Additionally, shifting to a new model may help them comply better with privacy laws all over the world, allowing the company to kill two birds with one stone. Echoing this sentiment, Kayda stated in an interview, “As we’re continuing to work on the app, now, we realise that by allowing access to those unfiltered models, it’s just hard to make that experience really safe for everyone…and we’re just not going to allow users to have unfiltered conversations, even if they’re romantic relations.”
However, with this wide-ranging change, it seems that Luka has alienated the most dedicated part of its audience. While the ethics of engaging in ERP with an AI chatbot is a matter for another day, the fact remains that many users felt they had lost a friend and confidant.
With the latest update, and Luka cleansing any mention of ERP, it seems a lot of users have been left stranded. However, if past occurrences are anything to go by, it is clear that Replika will recover from this lobotomy. However, the question is whether the community will recover from having their companions taken away time and again?
