The single biggest mistake that blinds us to the possibility of sentient machines is belief that human beings have something called a “soul”. This idea originates in our culture with the Greeks but it can be found in most civilisations. Only find a way to abandon it and the possibility of sentient machines becomes entirely plausible, not least because we are living embodiments of exactly that phenomenon.
Mind is the body in its inside-looking-outness. Certain kinds of bodies – not, notice, exclusively human bodies, and most definitely sufficiently elaborate cyber-bodies – situated in certain environments that stimulate them in suitable ways, will automatically generate and exhibit minds. Can minds survive the deaths of their bodies? Human minds cannot, but cyber-minds could conceivably be transferred into other working bodies. And what of the soul? Je n’ai besoin de cette hypothèse là.
People interested in the relationship between religion and science used to write about “The God of the gaps” in reference to religion’s always-futile attempt to argue that what science couldn’t explain God was responsible for, but the gaps kept shrinking until they became invisible. Now we have “The Soul of the gaps” to explain, so to speak, the things that minds can do that we cannot explain with reference to bodies, but these gaps are also shrinking at a very fast rate and will soon be invisible. Many human beings are already much less sophisticated than machines, and many human beings cannot pass Turing tests that machines can sometimes pass.
What we like to call “the soul” is no more than the subjective experience that automatically accompanies bodies of a certain kind of sophistication that have enjoyed a suitably stimulating environment for sufficiently long. For new-born babies such self-awareness is rudimentary and almost negligible, but given suitable nurture it can become extensive. For sufferers from degenerative brain diseases like Alzheimer’s the process goes into reverse and the sense of self gradually fades, never to return.
We may want the soul to be some sort of permanent repository for the self that can predate birth and survive mental degeneration and death, but it is just a dream.