|
Listen to this story
|
When we talk about AI, the intersection of temperature and Large Language Models (LLMs) may seem like an unusual pairing at first glance. After all, LLMs such as ChatGPT or Bard and its successors are complex algorithms designed for generating text, while temperature is a term we usually associate with thermodynamics. However, in the context of LLMs, temperature plays a vital role in fine-tuning the behavior of these models.
Temperature, in the context of LLMs, is a hyper-parameter used to modulate the randomness and creativity of generated text. It’s a concept borrowed from statistical physics and is integrated into the functioning of LLMs like GPT. This parameter allows users to adjust the balance between creativity and coherence when generating text.
Higher temperatures introduce more randomness, resulting in creative yet potentially less coherent output. Lower temperatures, on the other hand, yield more deterministic and focused responses, emphasising coherence over creativity.
Hot and cold models
Imagine using a high-temperature setting with an AI model to generate responses to a fictional scenario: “Describe a day in the life of an intelligent octopus.”
With the temperature set high, the AI might generate a response like:
“In a world where aquatic creatures acquired the power of human intellect, Octavia the octopus spent her days engaging in philosophical debates with her fellow marine inhabitants, pondering the mysteries of the deep.”
Here, the high temperature allows for an imaginative narrative, with the octopus taking on a humanoid level of intellect.
Now, let’s switch to a low-temperature setting and revisit the same scenario. With the temperature lowered, the AI generates a more grounded response:
“Octavia, the intelligent octopus, thrived in her underwater habitat. She communicated with precision, using a complex system of signals to coordinate hunting and navigation.”
In this case, the output is more focused and logical, emphasising the coherence of the story and the octopus’s natural behaviour.
It is important to note that adjusting the temperature does not alter the parameters of the original model. As OpenAI explains, “Temperature is a measure of how often the model outputs a less likely token. Higher the temperature, more random (and usually creative) the output. This, however, is not the same as “truthfulness”. For most factual use cases such as data extraction, and truthful Q&A, the temperature of 0 is best.”
Temperature just gives users more control over the creativity and stubbornness of the model’s output, which might be ideal for several different use cases.
How to adjust the temperature on ChatGPT
It is clearly visible that “setting the temperature” for a chatbot might actually be very beneficial for anyone using it. By tweaking the temperature, the model can cater to our specific needs. How do we do it? When it comes to ChatGPT, it is quite easy.
Just after you give a prompt on ChatGPT, and add “Set the temperature to 0.1” for a direct, less creative, and expected answer. Or write, “Set the temperature to 0.8” for a more creative response.
The temperature settings range from 0 to 1, and finding the right balance is the key. For straight answers, just type 0. Or if you want to be a little more creative, let it go till 1 and even beyond.
This begs the question – if one wants to keep an AI model from hallucinating in its responses, shouldn’t the temperature always be set to 0? Well, a user on Reddit explains that it depends on the training data of the model and the model does not get entirely deterministic even if the user sets 0 as the temperature.
Similarly, Bard also allows users to set the temperature range from 0 to 2. Unlike ChatGPT, Bard’s temperature also affects how detailed its responses get. It says, “you can also use the temperature setting to control the length of the text I generate. For example, if you set the temperature to 0.1, I will generate a short and concise response. If you set the temperature to 2, I will generate a longer and more detailed response.”
Can ChatGPT change its own temperature?
What if ChatGPT turns this temperature dial on its own and conjures hallucinatory mayhem? It seems like the times when the model starts hallucinating, it must be that it has decided to tweak the temperature knob itself.
When asked, ChatGPT assured that it’s not the mastermind behind this dial, claiming it lacks the self-awareness to twist it. “Users can specify the temperature they want when interacting with me, but I do not independently change this setting. It is up to the user to adjust the temperature setting to achieve the desired response style.”
Unfortunately, this means setting the temperature is also not the answer to prevent ChatGPT hallucinations.
