Sorry, aYrftDcJl9gp4LIqLhPqwLJrfgHC2lI- is the OpenID that Yahoo gave me by default before I made it use a sensible one.

I’m extremely interested in AI, having studied at university at a postgrad level. I’m still very pessimistic about it though. People have been saying the same things since the beginning and there really isn’t much progress – it’s an incredibly hard problem. I agree that doesn’t mean there will never be any and SOME people should occasionally debate things like the singularity because the possible consequences are enormous, even if the probability is extremely low. There is a similar situation for some of the frightening advances in nanotechnology that have been postulated.

Also, imagining what a hypothetical super-intelligence could do to help humanity may well lead to very promising research directions for domain-specific AI.

My last thought for this thread is that there are already a lot of intelligences way beyond the average human intelligence on the planet. They mostly sit around pondering the meaning of the universe via theoretical physics or some other problems in advanced abstract mathematics. Very occasionally something they do is picked up on by slightly less advanced intelligences and turned into technologies that do make changes to the world – usually positive but occasionally extremely destructive (Hiroshima?!). Most of the time, the world is competely ignorant of their thoughts and actions and carries on unchanged. Might not a similar thing happen with artificial super-intelligences. Is it possible that they may be capable of doing all sorts of things for or against humanity but most of the time they just aren’t interested – a bit like us with ants?

Mark

P.S. Thanks for writing about so many diverse and interesting topics on this blog, it’s a really good read.

Uh. Made me think. Thanks for the link.

Uh. Made me think. Thanks for the link.

>"We really don't understand intelligence and the field of AI has really only made baby steps since its inception."

That may be true, but it doesn’t mean that AI can never progress to the level of superhuman intelligence. Nor does it mean that progress to superhuman levels is necessarily going to take many, many decades.

>"The only way we can possibly speed up this process is if the machines start learning for themselves and evolving their own representations"

I believe many AI researchers will agree with you here, and will claim that they are already working on such systems. Other AI researchers might say that other routes are also worth exploring.

Because I can’t rule out in principle the possibility that researchers might stumble upon significantly improved systems or algorithms, I believe it’s well worth us trying to think ahead about that very possibility. I expect that’s what a lot of the discussion at Singularity Summit 2008 will be about.

>"If there were a sentient machine that was more intelligent than us then might it just NOT design a more intelligent machine which would eventually see its creator as a waste of resources and destroy it."

The research program of “Friendly AI” is designed to seek to avoid that possibility, by means of ensuring that the AI which eventually reaches superhuman intelligence will have friendliness towards humans built into it at a very deep level. It’s a tough challenge, but it has some plausibility.

// dw2-0

>"evolution does not favor intelligence by itself. It is just human arrogance to think that human intelligence is the perfection – and an inevitable result – of evolution."

I completely agree. My belief that we should pay serious attention to the possibility of the Singularity (with its very significant spurt in intelligence) doesn’t depend on any idea that evolution naturally leads to more intelligence. Instead, I see other more concrete factors at work, such as regular increases in hardware power, and (occasionally!) improved software algorithms. The risk is that these factors will result in the creation of superhuman intelligence.

This spurt of intelligence may or may not have been intended, by the researchers whose work resulted in this breakthrough. For an analogy, consider people introducing animal or plant species from one part of the world, to another part. Sometimes there are unexpected side-effects, with drastic consequences. We could see something similar from researchers who are merely experimenting with new algorithms, without realising their full consequences.

>"assuming that the intelligence explosion singularity happens someday in the connected computing network we are building, wouldn't it cause a bandwidth and power-consumption explosion too, bringing down the whole system?"

That’s one possible outcome. But another is that Singularity-level intelligence will (almost instantaneously) figure out radically improved new communications protocols that reduce the actual bulk of data flow, and reduce power consumption.

>"assuming that the intelligence explosion singularity happens someday in the connected computing network we are building"

By the way, that’s only one of five possible ways in which the Singularity could occur – according to the most recent paper by Vernor Vinge, “Signs of the Singularity”.

// dw2-0

The two main reasons for this as I see them are:

1) We really don’t understand intelligence and the field of AI has really only made baby steps since its inception. The only way we can possibly speed up this process is if the machines start learning for themselves and evolving their own representations. Machine learing is progressing but I haven’t seen anything like a starting point for this process appearing (not to say that it won’t, but software would have to get us there while faster hardware never can, it would just speed up the onset of the singularity once the process began).

2) If there were a sentient machine that was more intelligent than us then might it just NOT design a more intelligent machine which would eventually see its creator as a waste of resources and destroy it. i.e. It’s quite likely that a machine smarter than humans would be smart enough not to bring about its own destruction, since most humans are already smart enough for this (a few hard AI nuts not included).

Personally I believe that progress in AI is likely to remain domain specific for longer than any of the predictions you’ve given here. Researchers in the field have to be wildly optimistic to get any funding.

There is one fundamental flaw in this “intelligence explosion” logic, though. That is, evolution does not favor intelligence by itself.

It is just human arrogance to think that human intelligence is the perfection – and an inevitable result – of evolution. In fact, if you look at the natural world, all kinds of bacteria and insects are much more successful than the human race (in terms of absolute numbers or biomass). And if human race commits a mass suicide by nuclear holocaust, cockroaches might still survive and continue happily their non-intelligent evolution. And finally, human intelligence does not improve by evolution, unless it helps the individual’s chances of survival and reproduction: http://www.imdb.com/title/tt0387808.

In evolutionary systems, whether biological or AI, intelligence increases *only* if it helps the evolutionary unit’s chances of survival and reproduction.

And this is the key.

In computing networks, does it?

–

Ps. assuming that the intelligence explosion singularity happens someday in the connected computing network we are building, wouldn’t it cause a bandwidth and power-consumption explosion too, bringing down the whole system? Doesn’t this mean that the first beings on this planet of higher intelligence/consciousness are destined to a very short and miserable life? And doesn’t it mean that we wouldn’t even know that the singularity happened, and the news gets reported only as “The Great Internet collapse of 2046″. Hehe.

>"I find this to be a singularly unappealing future. Maybe I read too many scifi books as a kid — or maybe it‘s from watching the movie Colossus: The Forbin Project at an impressionable age. But either way, the idea of a sentient computer more capable than humans is about as threatening as a Terminator movie…”

If the Singularity goes bad, I doubt the outcome will be remotely like any of these Holywood movies – in which humans still manage to have a chance against malignant super-intelligent machines, by making (what shall I say?) a super-human effort. The outcome will probably be much, much worse. The machines (by dint of rapid self-improvement) will be way more intelligent and powerful than even the most powerful Hollywood actor. It will be game over, very quickly indeed. That’s Singularity Hell.

>"Unlike gazillionaire…"

If the Singularity goes well, it won’t just be the wealthy who benefit. Radically improved AI could in principle devise and manufacture very low cost super-drugs that will cure diseases worldwide. They could also design weather control mechanisms, to safely defuse hurricanes. Etc. That’s a hint of Singularity Heaven.

>"…Kurzweil"

Kurzweil is a populist and sometimes engaging writer but (needless to say) he’s not to everyone’s taste. Kurzweil is nominated by Washington Post writer Joel Garreau as being the iconic spokesperson for “The Heaven Scenario” prediction for what the radical development and application of new technology can achieve. This is in Garreau’s 2006 book “Radical evolution – the promise and peril of enhancing our minds, our bodies – and what it means to be human”. Personally, I find Garreau considerably more convincing than Kurzweil.

Garreau nominates Bill Joy of Sun Microsystems as the iconic spokeperson for “The Hell Scenario”, and virtual reality pioneer Jaron Lanier for “The Prevail Scenario”, which is a sort of (not completely convincing) middle path.

// dw2-0

But either way, the idea of a sentient computer more capable than humans is about as threatening as a Terminator movie. And, alas, a lot more imminent.