Agreed, I do expect AIs to become able to review the operation of their own source code, to notice bugs in it, and to fix them.

It will be slightly similar to the many “self reprogramming” methods often employed by humans.

Of course, once an AI improves itself by fixing bug A, it will be more likely to be able to detect and fix bug B too (where bug B is harder than bug A). And then, in short measure, bugs C, D, E… where each on the list requires even more intelligence to fix. The result could be a huge jump in the intelligence of the AI in just a short period of time (one afternoon?)

I didn’t say that the advances in what you call “narrow AI” were not relevant to the problem of general AI, or these advances would not help with “general AI”, I merely said that I did not consider these to be advances in AI (I consider them more to be advances in computer science or engineering). To use your flying analogy: the advances in the technology of the internal combustion engine towards the end of the eighteenth century were not advances in aeronautics, but they were extremely useful for the development of the airplane. I’m making more than a semantic point here: I think it is unhelpful for the progress of AI to claim advances in related fields to be advances in AI.

Note also that I’m not saying that I don’t believe that there will be advances in AI. On the contrary I believe, in the course of time, there will be real and significant advances in “general AI”. I just don’t believe that these advances will be made in the next decade.

Whilst I would agree that the existence of bugs doesn’t make a piece of software totally unreliable, the potential for any application or system crash and emitting of incorrect output is still there, which is why safety critical systems have multiple redundancy in the hope that the same bug isn’t replicated across all systems, as in aircraft. I suppose containing a minor error and rebooting that aspect of the logic is a reasonable approach, it’s quite like some automotive systems that continually reboot to ensure a clean slate, but losing a connection in the AI brain could lead to erroneous output, even if the whole system didn’t crash. Would we think that an AI with such an error is trusted, or would we think that it had a logical mental disease?

I guess one question that always rattles around in my mind is “why are we trying to recreate the human mind anyway?” We have billions of those already and they are very suited to the environment that they exist in. Why don’t we see more research on the type of AI that we’ll need for situations where a human is the wrong solution? Perhaps there is tons of this type of research and I miss it because it’s not my field, or perhaps it scares people to think that an alternative intelligence without human values might be dangerous.

I personally also really don’t think that logic by itself can lead to a system that can evolve human-like imagination, feelings or personality, nor that the human mind can be reduced to being a machine. It has elementary parts, but the constant rebuilding and evolving of information doesn’t really follow any logical rules that can be programmed. The structure of the brain depends on what happens to us during the day and how we interpret it according to the situation. That defies logic most of the time and is constantly evolving and changing. For example, I don’t like some things I liked when I was younger, but some I do. Personal taste is not logical. I don’t fancy a cheese sandwich today, but tomorrow I might if I see one.

You can build something that appears to be human, but what is the point of that? Why chase an goal that doesn’t actually provide us with more than we have already? Don’t we want an alternative intelligence that complements our own, not replacing it?

I’m coming across as very negative on this, which wasn’t my intention. What I don’t want is AI in products so that they have their own personality, but a better understanding of my own wishes and desires in how that product should interact with me. I don’t think that needs AI to achieve that goal, but instead better design.

Paul