Build a powerful computer, it becomes self-aware and turns evil. We've seen this theme in many movies including 2001: A Space Odyssey, War Games and Terminator 3. Computer scientists as Frankensteins, building monsters they cannot control.
In 1982, Disney put together a TV special that tried to argue against the computers as monster theme. They could have used a better title than "Computers are People, Too!" and avoided pushing their new movie Tron about an evil computer.
Colossus did scare me as a kid but when I grew up I realized computers, as powerful as they get, don't become self-aware or inherently evil and they can always be rebooted or unplugged. Bad people can use computers in evil ways but computers themselves are just tools not the perpetrators.
I bring this up because of the new movie Stealth opening today in the States about a plane controlled by a computer that becomes self-aware and starts destroying stuff. Scaring a new generation about the evils of computer science.
I think it's more like "Scaring a new generation about the evils of mad science, again using the hot topic of computers." It's hard to view computer science as especially put upon in this way, when your post already mentions Frankenstein. Doesn't this scare us all about the evils of biology?
ReplyDeleteNot to nit-pick, but the computer in "War Games" was not evil--it was simply programmed to develop a strategy to win a nuclear war. In the end it concluded, "Strange Game... the only winning move is not to play", and then proceeds to request a "nice game of chess". Though in the run up to this revelation, it has preparing the missiles it controlled for launch, it ultimately chose not to do so. I would say that this represents a triumph of computers and rationality over the typical aggression of humans.
ReplyDeleteOh, come on, Lance. You didn't go into CS, as all the rest of us did, because you dreamed of building evil self-aware computers? They said it was impossible. Fools! Soon they shall know that Shor's algorithm, Primes in P, and L=SL were all just stepping-stones in our secret plan to build the Polynomial-Time Quantum Destructo-Bot, and that once it's completed, we shall rule zee world! BWAHAHAHAHA!
ReplyDeleteToday we have "script kiddies" who create millions of dollars worth of damage by reusing and re-releasing computer viruses.
ReplyDeleteImagine a civilization so advanced that their brain capacities are orders of magnitude larger than ours (perhaps through augmentation). It would only take one such script kid from that civilization to create a lot of problems for us. Consider this a variation on the alien fear. This version is more scary, because appeals to "such an advanced civilization would be very ethical" wouldn't apply.
At some point, though, we might consider advanced intelligence a WMD that must be controlled.
Number five is alive... and he's got cluster bombs!
ReplyDelete(Maybe not directly related to what you wanted to point out) I guess unplugging a computer is not very easy in every situation. One cannot turn a mobile robot off if it is moving too fast. It may hurt his hand. Or even more importantly, one cannot unplug a distributed computer.
ReplyDeleteBryce, your post set a new world-record at missing-the-sarcasm.
ReplyDeleteDaniel
Lance is of course right that within our times, computers simply aren't aware or evil. But one day in the far future, computers really might be smarter then people. I'm don't know whether those computers will be "evil", or even whether the question makes sense. But when it does happen, computers will be in full control of the planet; the human era will be over.
ReplyDeleteThe idea of unplugging or rebooting a sentient computer network is nonsense. I imagine that crocodiles likewise conclude (at some primitive level) that they don't need to worry about people, because people can always be eaten.
It could be an impossible artistic challenge to devise logical science fiction about hyper-intelligence, either the alien kind or the computer kind. People want a plot that somehow revolves around people, not a plot in which people don't matter. The only science fiction that I really like responds to the challenge with absurdity. My personal favorite is The Cyberiad, especially the last story, "Prince Ferrix and the Princess Crystal".
I'm sorry Greg Kuperberg, but you're way off the mark with this one. Okay you may have a point that certain types of computer may be very difficult to shut down, the Internet springs to mind. But to say that this supports a belief that computers will one day become intelligent (more so than humans) is preposterous.
ReplyDeleteLet me set the record straight; there will NEVER be a Computer more intelligent than a human. The most powerful modern computer is still no more "clever" than the first computers ever made. Bigger, Faster, more complex, more useful but not intelligent. Some quantum computing (theory) does suggest slightly different paths computers might take, but there isn�t even the tiniest shred of evidence to suggest a computer might ever become self aware.
It�s an interesting thought I�ll admit and quite an appealing one in terms of movie goers but I�m afraid the highest level of self awareness a computer is ever going to reach is that of an abacus.
When I argued that of course people would never be able to unplug a superior intelligence, I did not mean that as an argument that computers will one day be smarter than people. My point is that if they one day are, it will be the end of the show for Homo Sapiens. Whether the computers would be better or worse than their obsolete progenitors is hard to say. Who knows, maybe they would be a billion times more benevolent than we are.
ReplyDeleteI don't know how you can know that computers will never be more intelligent than people. It is not part of the forseeable future, but that isn't all that long these days. Do you think that human intelligence is supernatural? If not, why can't it be invented?
Let me set the record straight; there will NEVER be a Computer more intelligent than a human.
ReplyDeleteTo me in many ways they already are. They do better math than we do, they are better at playing chess than we are, they have already proven theorems and discovered physical gadgets that we didn't know off (including some that have been patented), they can create art, some of which is quite outstanding.
We even wrote a program that was fully self-aware. It knew that any action it took would impact the system and thus it would have to take itself into account in its own analysis. It even surprised its creators and came up with a wholy unanticipated answer.
Alex Lopez-Ortiz