Dr. Stuart Russell, a distinguished AI researcher and computer scientist at UC Berkeley, believes there is a fundamental and potentially civilization-ending shortcoming in the “standard model” of AI, which is taught (and Dr. Russell wrote the main textbook) and applied virtually everywhere. Dr. Russell’s new book, Human Compatible: Artificial Intelligence and the Problem of Control, argues that unless we re-think the building blocks of AI, the arrival of superhuman AI may become the “last event in human history.”
That may sound a bit wild-eyed, but Human Compatible is a carefully written explanation of the concepts underlying AI as well as the history of their development. If you want to understand how fast AI is developing and why the technology is so dangerous, Human Compatible is your guide, literally starting with Aristotle and closing with OpenAI Five’s Dota 2 triumph.
Stuart’s aim is help non-technologists grasp why AI systems must be designed not simply to fulfill “objectives” assigned to them, the so-called “Standard Model” in AI development today, but to operate so “that machines will necessarily defer to humans: they will ask permission, they will accept correction, and they will allow themselves to be switched off.”
What Dr. Russell refers to as the Standard Model has a simple definition: “machines are intelligent to the extent that their actions can be expected to achieve their objectives.” Dr. Russell worries that one day super powerful AI-based systems may decide that humans stand in the way of their objectives, which would place humanity at risk.
To forestall that possibility, he wants to change the Standard Model along these lines: “Machines are beneficial to the extent their actions can be expected to achieve our objectives.” He has already made that leap in the forthcoming fourth edition of his AI textbook Artificial Intelligence – A Modern Approach. But as discussed in this interview with TechCrunch, research has barely begun on what that shift really means, or how best to make it happen.
How big is the AI threat? Today’s systems are designed to optimize for “objectives,” which are strictly speaking what the creator of the algorithm sets up. There is no awareness of unintended effects that might cause the AI to pause and question whether the objective is really desirable or anticipates unexpressed human preferences. Dr. Russell’s favorite example is the content selection algorithms in social media.
“Like any rational entity, the algorithms learn how to modify the state of its environment — in this case, the user’s mind — in order to maximize its own reward,” he writes. “The consequences include the resurgence of fascism, the dissolution of the social contracts that underpin democracies around the world, and potentially the end of the European Union and NATO. Not bad for a few lines of code. Now imagine what a really intelligent algorithm would be able to do.” Dr. Russell cites the legendary MIT mathematician, Norbert Wiener, who in a 1960 paper warned, “we had better be quite sure that the purpose put into the machine is the purpose we really desire.”
Dr. Russell’s objective in the first half of the book is to educate readers on the conceptual building blocks of AI as well as why human-level and super-intelligent AI remain such hard technical challenges. Dr. Russell delivers a brief history of intellectual underpinnings that begins with Aristotle’s definition of intelligence and layers on later concepts of utility and probability to reach the research of Judea Pearl, (2011 Turing Award winner) who developed “uncertain reasoning,” which issued in the era of modern AI.
Centuries of mathematical and conceptual work made it possible for AI algorithms today to come up with a good (not perfect) answer to the question, “What time should I leave for the San Francisco airport? (I just asked Alexa, and she told me I should allow “about 35 minutes, based on current traffic.”)
If Alexa can do that, then when will a general purpose, super-intelligent AI be able to take on more existential matters, like ending world hunger, stabilizing the climate, or putting Elon on Mars? That’s a very hard question to answer, and Dr. Russell reluctantly offers in his book that he guesses that human-level AI will emerge in his grandchildrens’ lifetimes, maybe sooner, which places him in the middle of the road among AI experts.
At the same time, he acknowledges that we live in a world where AI breakthroughs are almost as common as billion-dollar startup valuations. Back in April, OpenAI Five defeated the world’s top pro team in the game Dota 2, an achievement that overshadowed even Google’s Alpha Zero wins in the complex strategy game Go. Game playing AIs rely on a concept called “reinforcement learning,” which basically means that the AI teaches itself how to play and win by playing against itself — 45,000 years of self-game play over ten months in the case of Open AI Five’s mastery of Dota 2.
Impressive as that is, Dr. Russell observes, there are “sharp boundaries” around what Alpha Zero or OpenAI Five can do because Go and Dota 2 are “discrete, observable, two-player games with known rules.” The world and what we know about it is vastly more complex and anything but rule-bound. Dr. Russell suggests that superhuman AI is probably decades away but at that the same time points out several times in his book that in 1933 Leo Szilard figured out how to create a neutron-induced nuclear chain reaction the day after Lord Ernest Rutherford declared that such a development was near impossible. “Betting against human ingenuity is foolhardy,” Dr. Russell notes.
Human Compatible is the indispensable book on AI. A pre-eminent AI scholar and researcher not only rings the alarm bell — anyone can do that, and many have — he also makes two-and-a-half vital contributions few others could. First, he explains the conceptual and historical building blocks of AI, which give the lay reader a vital grasp of AI, which in turn makes clear how AI could become dangerous.
Second, he suggests the remedy — a wholly new approach to AI to ensure that AIs behave more like Jeeves than Skynet. That strikes this reader as an enormous contribution, and one that we have to hope takes route in research facilities as well as big tech companies.
Where Dr. Russell does not quite deliver is what policy wonks want to hear: what should policymakers do to forestall the AI threat? He goes into fledgling protections in GDPR related to AI and industry groups like Partnership on AI, but stops well short of making concrete recommendations on exactly how humanity can protect itself, assuming that the “dudes chugging Red Bull in the lab,” will be the last to see their catastrophes in the making. But perhaps that job should fall to readers of the book.
Dr. Russell and I discuss that issue and more in this interview, conducted in his office at Berkeley on Sept. 3.
Comment