SILICON Valley’s feverish embrace of large language models (LLMs) shows no sign of letting up. Google is its chatbot Bard into every one of its services, while OpenAI is imbuing its own offering, ChatGPT, , such as the ability to “see” and “speak”, envisaging a new kind of personal assistant. But deep mysteries remain about how these tools function: what is really going on behind their shiny interfaces, which tasks are they truly good at and how might they fail? Should we really be betting the house on technology with so many unknowns?
There are still large debates about what, exactly, these complex programs are doing. In February, sci-fi author Ted Chiang a viral piece suggesting LLMs like ChatGPT could be compared to compression algorithms, which allow images or music to be squeezed into a JPEG or MP3 to save space. Except here, Chiang said, the LLMs were effectively compressing the entire internet, like a “blurry JPEG of the web”. The analogy received a mixed reception from researchers: some praised it for its insight, and others accused it of oversimplification.
Advertisement
It turns out there is a deep connection between LLMs and compression, as shown by a from a team at Google Deepmind, but you would have to be immersed in academia to know it. These tools, the researchers showed, do compression in the same way as JPEGs and MP3s, as Chiang suggested – they are shrinking the data into something more compact. But they also showed compression algorithms can work the other way, too, as LLMs, predicting the next word or number in a sequence. For instance, if you give the JPEG algorithm half of an image, it can predict what pixel would come next better than random noise.
This work was met with surprise even from AI researchers, for some because they hadn’t come across the idea, and for others because they it was so obvious. This may seem like an obscure academic warren that I have fallen down, but it highlights an important problem.
Many researchers working in AI don’t fully understand the systems they work on, for reasons of both fundamental mystery and for how relatively young the field is. If researchers at a top AI lab are still unearthing new insights, then should we be trusting these models with so much responsibility so quickly?
The nature of LLMs and how their actions are interpreted is only part of the mystery. While OpenAI will happily that GPT-4 “exhibits human-level performance on various professional and academic benchmarks”, it is still unclear exactly how the system performs with tasks it hasn’t seen before.
On their surface, as most AI scientists will tell you, LLMs are next-word prediction machines. By just trying to find the next most likely word in a sequence, they appear to display the power to reason like a human. But from researchers at Princeton University suggests many cases of what appears to be reasoning are much less exciting and more like what these models were designed to do: next-word prediction.
For instance, when they asked GPT-4 to multiply a number by 1.8 and add 32, it got the answer right about half the time, but when those numbers are tweaked even slightly, it never gets the answer correct. That is because the first formula is the conversion of centigrade to Fahrenheit. GPT-4 can answer this correctly because it has seen that pattern many times, but when it comes to abstracting and applying this logic to similar problems that it has never seen, something even school kids are able to do, it fails.
For this reason, researchers that we should be cautious about using LLMs for problems they are unlikely to have seen before. But the millions of people that use tools like ChatGPT every day aren’t aware of this imbalance in its problem-solving abilities, and why should they be? There are no warnings about this on OpenAI’s website, which just states that “ChatGPT may produce inaccurate information about people, places, or facts”.
This also hints that OpenAI’s suggestion of “human-level performance” on benchmarks might be less impressive than it first seems. If these benchmarks are made mainly of high-probability events, then the LLMs’ general problem-solving abilities might be worse than they first appear. The Princeton authors suggest we might need to rethink how we assess LLMs and design tests that take into account how these models actually work.
Of course, these tools are still useful – many tedious tasks are high-probability, frequently occurring problems. But if we do integrate LLMs into every aspect of our lives, then it would serve us, and the tools’ creators, well to spend more time thinking about how they work and might fail.
Alex’s week
What I’m reading
The Paris Review Interviews, Volume II, insights from the most decorated writers of the 20th century. There are a million gems in each interview.
What I’m watching
I’ve been catching up on 70s horror flicks. Jaws hasn’t aged a second, and Picnic at Hanging Rock was intriguing.
What I’m working on
Like much of the 91av team, I’ve been working on various bits and pieces for this year’s 91av Live.
Alex Wilkins is a 91av reporter covering artificial intelligence, physics and space. Artificially intelligent is a column that cuts through the hype and looks at what AI is really capable of and what it means for us. You can follow him @AlexWilkins22
