SuperIntelligence: A book Review
“The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.” Eliezer Yudkowsky
AI is a very hot topic today, mainly because of the revolution in pattern recognition which comes under the heading of Deep Learning. Hollywood conditioned us to think of Artificial intelligence as a dangerous technology that will bring forth the end of human civilization. If you are a fan of these ideas, then Nick Bostrom’s Superintelligence will not disappoint. In his book, the author asks the question “What would happen when machines achieve a level of intelligence far beyond anything that humans are capable of?”, he calls this ability Superintelligence.
Before we get to the review, a little background is in order. First, there is a wide consensus today among computer scientists that the brain is nothing more than a computer and it is possible to create an artificial version of the brain by programming a computer with the right algorithm. This version of Artificial intelligence is known as Strong AI to be distinguished from Weak AI which posits that the computer will only be able to simulate thinking without actually thinking for itself.
Secondly, there is a very popular computer science theory known as the Singularity that has been best advocated by Ray Kurzweil. The latter believes that we will reach Strong AI in 2030 at which point we will be able to download our brains into computers and then we will live forever in a virtual world.
Now Bostrom is a fan of Strong AI. He takes for granted that the algorithm or program to create human level intelligence (or Artificial General Intelligence) could be created by making an ultra high resolution scan of all the interconnections that neurons make in the brain. This will require a scanning technology and a splicing technology that is beyond today’s technologies, but not theoretically far fetched in the future.
Bostrom isn’t much concerned about when we will reach the Singularity - he takes it as a given - whether it be in 10 years, 50 years or many centuries; we will get there. He doesn’t see any problem with downloading minds or for the AI achieving consciousness and creating artificial human minds. But when we do get to the “Singularity” or to a Superintelligence there’s going to be trouble if we haven’t figured out how to control the AI.
For Bostrom, the superintelligence argument comes from the fact that the computer with General Human intelligence can always increase its intelligence following an exponential process like Moore’s Law. This seems like an intrinsic property of hardware.
Thus faced with the prospect of a Superintelligence, humanity’s biggest concern should be how to control it. Otherwise it might pose an existential danger for us. His argument on this front can be summarized as follows:
1- A superintelligence by its own virtue will form a “Singleton” which is akin to a world government or dictatorship which controls the world. This will be usually the first project that creates an artificial general intelligence.
2- The orthogonality thesis: “any level of intelligence could be combined with any final goal”. Which means that while the Superintelligence might have some wise and benevolent goals, it can also have some goals that go against human interests.
3- The instrumental convergence thesis: “superintelligent agents having any of a wide range of final goals will nevertheless pursue similar intermediary goals because they have common instrumental reasons to do so”. These subgoals could include resource acquisition.
4- The resources that the superintelligence might want to acquire might be resources that human beings need for their survival.
Therefore: A Superintelligence whose final (and intermediate) goals are not controlled might destroy humanity by an open ended quest for resource acquisition.
This argument is best exemplified by the paperclip maximizer. This thought experiment presents a machine who was given the directive to maximize the production of paper clips. So once it has achieved Superintelligence it will try to convert most parts of the visible universe into paperclips. He also gives an example where a super intelligence tasked with maximizing human happiness. This one will just inject all humans with drugs to make them high, thus achieving its goal the fastest possible way.
This problem of controlling the AI, the author calls the Control Problem and spends a big part of the book discussing possible solutions to it. He considers scenarios where the AI is trapped in a simulation so that it doesn’t do any harm but also accepts the possibility that it will be very difficult to achieve and if done properly the AI might not be of much use to humans. He considers the prospects of creating some safety mechanism in the AI, some sort of “Off Switch”, but disregards this since the AI will figure out how to deactivate it. For him the best way to control the AI is to imbue it with positive values. One way to do it might be to start off with a brain simulation of a human being with positive values, but then he asks “How are we to determine what are positive values?”
I think this book has some major flaws. The most obvious one is taking for granted that Computers will achieve and surpass human level intelligence. If the past is anything to go by, this prediction has been proven wrong many times. This objection aside, the book spends a lot of time analyzing how an AI is going to mis-interpreting its prime objective and not enough realizing that the AI will understand what we mean with the objectives that we give it. Although the author does discuss this possibility, he finds a lot of difficulties in having an AI that actually does this. For him, the prime objective of the AI is to achieve its goal and therefore it will only care instrumentally about what the programmer meant by his objective, it will only care about achieving the goal as it understands it.
His model of the AI is sort of like an autistic savant, who is able to do a lot of very powerful intellectual feats but fails to grasp some common sense notions that normal intelligence human beings are able to handle well. Why he chose to model the AI this way, one can only speculate. He probably seems to be mixing what current AI understand their objectives given current programming language and uses that as a base for how future AI will interpret their objectives. But it seems reasonable that the level of programming that is required to program the initial AI will also have solved the problem of imbuing the AI with the ability to understand human meaning, this will be a given from its ability to understand human language (an AI-complete problem as the author likes to say).
The book is a tough read, not the least because of the technical language that the author uses that betrays his training as an analytical philosophers. Consider this exert: “An oracle constructed with domesticity motivation might also have goal content that disvalues the excessive use of resources in producing its answers.” However as one of the two books recommended by Bill Gates to learn about AI (The other being “The Master Algorithm”), it is a must read for anyone currently interested in the state of the art thinking about AI.
References:
http://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom
Comments
Post a Comment