In a future where each individual has access to their own personalised AI companion, OpenAI CEO Sam Altman believes we would need to redefine “privileged information.”

During a recent interview with MIT president Sally Kornbluth, Altman spoke on the privacy versus utility debate and where the tradeoff might be set in the near future. In particular, he proposed the hypothetical of a personalised AI that has access to nearly all of an individual’s personal data and whether it could potentially be used as evidence against in a court of law.

“That would be super helpful. But you can also imagine the privacy concerns that would present. How are we going to navigate the privacy versus utility versus safety tradeoffs or security tradeoffs that come with that? Do we need a new definition of ‘privileged information’ so that your AI companion never has to testify against you or can’t be subpoenaed by a court?” he asked.

Of course, Altman points out the advantages one could have on letting an AI train on their entire lives. But exactly where people will put limits on the tradeoff between privacy and utility, he says, will be a new thing for society to navigate.

Further, he implied that the current privacy debates were only the tip of the iceberg, with conversations on data privacy having the potential to go far when it comes to AI.

“There are all these things that we’ve had to negotiate with the internet, things about how we think about privacy, how we think about online ads, that when you intersect them with AI, become much higher stakes and much bigger trade-offs,” he said.

Interestingly, he also pointed out that these issues could be sorted out sooner if a way to separate reasoning engines from the data needed to train them is found.

“The only way we currently know how to do that (make a reasoning engine) is by training them on tons of data. I think we’ll look back and say that was a weird waste of resources because though it is true that GPT-4 can act as a database, it’s slow and expensive and doesn’t work very well,” he said.

However, if developers manage to separate the two so that the engine itself doesn’t store the data it is trained on, the debate surrounding the issue of privacy could be eased.