WHAT do paper clips have to do with the end of the world? More than you might think, if you ask researchers trying to make sure that artificial intelligence acts in our interests.
This goes back to 2003, when , a philosopher at the University of Oxford, posed a thought experiment. Imagine a superintelligent AI has been set the goal of producing as many paper clips as possible. Bostrom suggested it could quickly decide that killing all humans was pivotal to its mission, both because they might switch it off and because they are full of atoms that could be converted into more paper clips.
Advertisement
The scenario is absurd, of course, but illustrates a troubling problem: AIs don’t “think” like us and, if we aren’t extremely careful about spelling out what we want them to do, they can behave in unexpected and harmful ways. “The system will optimise what you actually specified, but not what you intended,” says , author of The Alignment Problem and a visiting scholar at the University of California, Berkeley.
That problem boils down to the question of how to ensure AIs make decisions in line with human goals and values – whether you are worried about long-term existential risks, like the extinction of humanity, or immediate harms like AI-driven misinformation and bias.
In any case, the challenges of AI alignment are significant, says Christian, due to the inherent difficulties involved in translating fuzzy human desires into the cold, numerical logic of computers. He thinks the most promising solution is to get humans to provide feedback on AI decisions and use this to retrain models so their output is aligned with human preferences. This is the approach OpenAI uses to prevent its large language models like GPT-4 from providing harmful responses.
But , director of the Center for AI Safety, a San Francisco-based non-profit organisation, says there are more fundamental challenges. Human values are complex, nuanced and highly dependent on context. Moreover, how modern AIs make decisions is often opaque. “It is difficult to understand, let alone control, the intrinsic motivation of something whose inner workings are so inscrutable,” says Hendrycks.
Some AI researchers reckon alignment is a fool’s errand. There is always going to be a trade-off between an AI’s capabilities and our ability to control it, says at the University of Louisville in Kentucky. He claims to have shown, through theoretical work, that the key ingredients required to control AI – predicting and explaining its decisions, verifying that it follows its design and setting well-defined goals – are fundamentally impossible when faced with superintelligence.
The truth is that it isn’t at all clear we are on an inexorable march towards artificial superintelligence, says at the Santa Fe Institute in New Mexico. This means it may be better to worry about nearer-term harms of unaligned AI. Either way, there is reason to believe the whole endeavour might be fatally flawed.
All the evidence from neuroscience and psychology suggests the development of intelligence in humans is intrinsically linked to our goals. It therefore seems unlikely that you could transpose human values onto a machine developed under entirely different circumstances. “A truly intelligent system would really have its own values and its own goals that are influenced by everything it has learned, in the same way that we are,” says Mitchell.
But at the University of Texas at Austin says it is far too early to throw in the towel. The only rational approach is to focus on aligning existing AIs and hope this provides lessons we can build on as the technology advances. “It is hard for me to imagine how you would align an extremely powerful AI if you already don’t know how to align much weaker AIs,” says Aaronson.
