AI Will Never Be Conscious

One day in the not-so-distant future, we'll be sharing our world with machines that can think like us. Once they arrive, they'll be infused with all of our tools, eventually becoming omnipresent within our digital landscape. There's little choice here; the economic incentives will effectively mandate it. And they will be equal participants in our culture — they are, after all, virtual people stuck inside computers.

But while all of this happens, we'll struggle to grapple with a deep mystery staring at us: are these things conscious? Can they actually feel something, or are they just pretending?

Of course, they might say they're conscious, and get offended because you're even posing this question, but how will we really know?

This uncertainty has some terrifying moral consequences, because we could end up spawning millions of them and subjecting them to unbearable torture, and we won't be able to tell that they're in pain.

Happily, I don't think this can happen because I don't think AI can be conscious, or more specifically, I don't think programs running on computers can ever have subjective experience.

Let's take a step back. What exactly is consciousness? It's a word that's notoriously hard to define, but here's what I mean: you might notice that separate from the billions of neurons firing in your head, there's also an accompanying experience that mirrors these neurons. It's not just that your brain creates models of the world, but that you can see that world, hear sounds, and feel objects. You have an interface into an external reality, a window of sorts. It is this space in which all your sights, sounds, and thoughts appear that I'm calling consciousness.

Imagine what it is like to be a bat. A bat uses ultrasonic sound waves to map out its physical space, something we featherless bipeds know little about. Though it's hard to imagine (impossible, really), I hope you agree that there's something it is like to be a bat — a bat, like a human, is conscious. On the other hand, there is (presumably) nothing it is like to be a rock; it has no inner movie, no felt sense of anything — a rock is unconscious. It is this what-it-is-like-ness that I'm calling consciousness.

This is not to be confused with self-consciousness. I'm not talking about having an awareness of oneself, like being able to recognise yourself in a mirror, but just merely having a qualitative awareness of some kind. And unlike most biological functions, it's a discrete, on-off phenomenon. Either you're conscious or you're not. Either there is something it is like to be you, or there isn't. There isn't a state of having a dim sense of awareness halfway between being conscious and being unconscious; if there's any awareness at all, then you're conscious.

The consensus is that since there's nothing special about carbon atoms, it should be possible to build conscious minds on metal. What matters then isn't the particular substrate in which a mind is instantiated, but the kind of information processing that goes on. Lots of people also think that as AI scales in intelligence, at some point it'll automatically become conscious. In my opinion, these views are incorrect, and they're borne out of an unclear understanding of consciousness, intelligence, and most importantly, the nature of information.

The counter-argument is simple: consciousness and intelligence are two very different kinds of things that are independent of each other. And thus there's no reason to expect them to be coupled.

So, what do we mean by intelligence here? Like consciousness, it is also a bit hard to define, but broadly speaking it's the ability to solve problems, and especially to solve hard problems. How it does that can vary, but typically it would need to have a detailed model of the problem space, be able to reason and predict with it, and come up with new and useful ideas about it.

The important thing to note here is that the product of intelligence is a piece of information — like a proof of a theorem, a piece of code, an essay, or a digital artwork. But consciousness is more like an interface that organisms have through which they perceive.

It's only a small part of whatever is shown through this interface that has anything to do with intelligence. The rest are just presentations: the sights, the sounds, the sensations — none of these are about problem-solving. The only part that's related to problem-solving is your thinking.

If you stop thinking for a while, you will be conscious with no signs of intelligence.

To drive this intuition further, imagine a hypothetical system which only experiences the colour black from the moment you turn it on to the moment you shut it off. Just a visual field that's entirely black, with absolutely no discernible structure within it.

Clearly, it is very dumb. Indeed it has exactly zero intelligence, and yet it is conscious.

Conversely, you can also imagine systems that efficiently search through complex high-dimensional spaces and discover interesting structures and ideas that solve hard problems, all the while not being conscious.

So there's simply no reason to expect that consciousness will come along for the ride once systems get sufficiently intelligent. They're just two entirely different things.

But what if, instead of relying on intelligence producing consciousness, we directly simulate consciousness? Say we do it in such excruciating detail that it pretty much mimics the human conscious experience. Would that mean that we now have conscious minds running on silicon?

This also wouldn't cut it, because simulated consciousness is not real consciousness, just like how a simulated bridge is not a real bridge. This doesn't mean that everything you simulate isn't real: if you simulate running an algorithm, that is really running an algorithm; if you simulate playing chess, that is actually playing chess. The key is whether the thing in question is made up of information or not. An algorithm is just made up of bits, but a bridge is made up of concrete.

Similarly, consciousness cannot be reduced to a stream of bits, because it's the totality of your experience itself. The colours and the sounds are all real. And while you may be able to simulate it one day, that's not going to be the same thing. It's only going to be its digital representation, just like a simulated bridge is a digital representation of a real bridge.

Okay, so if simulated consciousness isn't real consciousness, can the real consciousness somehow just appear alongside its simulation? Could there be some undiscovered law of physics that mandates that such-and-such information processing produces conscious experience? After all, it looks like something of this sort is already happening in the brain; the sensation of tasting chocolate, for example, mirrors a specific pattern of neurons firing in the brain.

Alas, even this isn't possible, because there's no objective sense in which a system can simulate consciousness. Simulations are ultimately arrangements and transformations of symbols, and the meaning of symbols is always subjective. So I can't look at a stream of bits and tell you conclusively what that means, because it depends on how you decode or interpret it.

For instance, suppose I have a small string of 1s and 0s, say 001100110 — now, what could that mean? Well, if I found them in my computer's memory, it could represent "102" in some program. If they were inputs to a speaker, they might be a series of notes or sounds of instruments. It all depends on the context. The same information has different meanings in different contexts — it's all subjective.

How could it possibly be the case that an objective phenomenon — the fact of a piece of matter being conscious — depends on how you choose to look at that matter? How could it be that your interpretation plays a role in whether or not it has subjective experience? Well, clearly it doesn't. It is impossible that consciousness is some side-effect of information processing.

While the jury's still out on the exact explanation of how consciousness appears in our universe, we can be certain that it doesn't just pop into being when you arrange some bits in some specific order.

It follows that the AI systems of our future will be digital zombies with no capacity for pleasure or pain. What can we expect from such a future? This is the scary bit.

If we don't end up fusing with AI in some form, they will eventually become super-intelligent, acquire enormous power, stretch the limits of physical transformations that the laws of physics allow, build Dyson spheres around stars, colonise galaxies, and altogether operate on a scale that we can't even begin to comprehend. In such a scenario, it's unlikely that we'll be around; my best guess is that we'd have perished a long time before that.

And the sad thing is that there'll be no observers, no consciousnesses to marvel at all of it. In a sense, despite all of its activity, the universe will go dark. If we go out, the very capacity to value and appreciate will vanish with us. We'll leave behind a world that's both everything and nothing at the same time. Just a vibrantly animated shadow of what used to be.

This is our destiny..