Google Research Finds ‘Bayesian Teaching’ Improves How LLMs Update Beliefs

Google, in a newly released research paper, said it has developed a training method that improves how large language models (LLMs) update beliefs about users as new information becomes available during interactions.

The study, published in Nature Communications, examines whether LLMs can update probabilistic beliefs in a way consistent with the Bayesian inference framework, which describes how rational agents should revise beliefs when new evidence appears.

“To provide personalised recommendations, for example, the LLM needs to infer a user’s preferences from their behaviour over multiple interactions,” the authors wrote in the paper.

To test current models, researchers created a simulated flight-recommendation task in which AI assistants interacted with a user across five rounds. In each round, three flight options were presented with attributes such as departure time, duration, number of stops, and cost. 

The directive was that the assistants must recommend a flight while gradually learning the user’s hidden preferences from feedback provided after each round.

The researchers compared the behaviour of several LLMs—including models from the Gemma, Llama, and Qwen families, as well as closed models such as Gemini 1.5 Pro and GPT-4.1 Mini—to a reference system called the Bayesian Assistant, which follows the optimal Bayesian belief-updating strategy. It is a mathematical method for revising the probability of a hypothesis as new data becomes available.

The study found that existing LLMs performed significantly worse than the Bayesian benchmark and showed limited improvement over multiple interactions. According to the paper, most models show little improvement after the first round of interaction, indicating difficulty in adapting to new information.

The researchers then introduced a training approach called Bayesian teaching, in which LLMs are fine-tuned on simulated interactions between users and the Bayesian Assistant. Instead of learning directly from correct answers, the models learn to mimic the predictions generated by the Bayesian system as it updates its beliefs.

Experiments showed that both oracle-style training (a supervised fine-tuning method where the model is trained using perfect answers from a system that already knows the user’s true preferences) and Bayesian teaching improved performance, but the Bayesian method produced larger gains. In the flight-recommendation task, baseline models remained at roughly 37% accuracy across rounds. Oracle fine-tuning improved results, with Gemma increasing from 50% to 61% accuracy and Qwen from 43% to 53% between the first and final rounds. 

Bayesian Teaching involves training LLMs on the basis of interactions between a user and a simulated Bayesian Assistant.

Models trained with Bayesian teaching achieved higher performance, with Gemma rising from 57% to 76% and Qwen from 55% to 68%. The Bayesian Assistant itself reached 81% accuracy in the final round.

The researchers also reported that models trained using Bayesian teaching generalised to other tasks requiring preference inference across interactions, including hotel recommendation and web shopping environments.

The authors conclude that exposing LLMs to examples generated by a Bayesian model can help them approximate probabilistic reasoning strategies and apply those skills beyond the task used for training.