# 2023-04-08 - Reason Is Fun - #-1 - AGI with David Deutsch [Official](https://reasonisfun.podbean.com/e/agi/) [YouTube](https://www.youtube.com/watch?v=jQnoxhoWhXE) [Apple Podcasts](https://podcasts.apple.com/us/podcast/1-agi-with-david-deutsch/id1681458005?i=1000608045345) Duration: 01:01:18 ## Transcript ### Lulie Tanett 00:00:00 - 00:00:47 Welcome to the Reason is Fun podcast. I'm your host, Lulie Tanett. Today I'm having a conversation with David Deutsch about AGI and epistemology. And it's more of a conversation than an interview, so I get a little bit interrupty at times. This episode might be a little rough around the edges, which is why I've called it episode minus one. And episode zero might also be a bit rough. So I appreciate your patience. And with that, let's get into it. A bunch of things are happening in the world right now regarding AI and people are panicking. And I wanted to know what you thought about that whole thing. ### David Deutsch 00:00:47 - 00:01:03 Yeah, my broad thought, with which I always reply when people ask me about this, is that AI is not the same as AGI. AGI is not a very advanced form of AI. ### Lulie Tanett 00:01:03 - 00:01:08 Will AI destroy the universe or just the world? ### David Deutsch 00:01:08 - 00:01:49 I think there's no more reason to think that AI will destroy the world than any other technology or than people in general. AGI, once we have it, will be just people. And they certainly have the potential to destroy the world. But we also have very deep knowledge about how to prevent that. So as long as we keep making progress, I don't think there's anything more effective we can do to ensure good outcomes in the future than to keep making progress. Certainly, if we don't make progress, we've guaranteed our doom. ### Lulie Tanett 00:01:51 - 00:02:50 Okay, suppose I am writing a reply to a Bayesian who is very worried about AI and is specifically worried that it will grow too fast and then it will gain intelligence and then do all sorts of bad, dangerous things and destroy the world. One of the things I find odd about questions like that is the way that people are focusing on a particular danger that is worrying them for some reason. And they ignore equally dangerous or worse possibilities that either are always around or have been around for a long time. So why do some people freak out? ### David Deutsch 00:02:50 - 00:03:11 I don't think that's too strong a phrase over AI risk and other people freak out over climate change risk. And now people are starting to freak out about nuclear war risk again. That hasn't been happening for like decades now. But I remember when it was the big thing. ### Lulie Tanett 00:03:12 - 00:03:16 Did you see a similar freak out in the last time this happened? ### David Deutsch 00:03:16 - 00:04:21 Absolutely. I mean, I think probably one of the biggest things that happened was that I think probably more than now it was a complete consensus that everybody was afraid. Let me think. I think everybody was appropriately afraid. Was it like the pandemic? It's unlike today. Was it like the pandemic freak out? Because you also got like people actually freaking out around the pandemic and doing all sorts of things like covering their door handles in copper and not knowing whether masks work or not and so on. And these battles with their friends about whether they leave their house and whether that is akin to killing people. The interesting thing, I think there's an interesting difference between being afraid of something like nuclear war or a pandemic and being afraid of something that one imagines like AI risk or climate risk. So those are both things that might happen in the future and different people might imagine different things about them. ### Lulie Tanett 00:04:21 - 00:04:23 You mean AGI risk in the future? ### David Deutsch 00:04:23 - 00:04:38 Well, there's AGI risk and nowadays there seems to be AI risk as well. I mean, there is AI risk. People are using it to scam people and to fake voices of Barack Obama and ring people's grandmothers and so on. ### Lulie Tanett 00:04:38 - 00:04:40 What about those? ### David Deutsch 00:04:40 - 00:05:18 Yes, well, those are real risks and you know, there's the electric car risk and the self-driving car risk and so on. That's not in the same league as having a greatly increased ability to scam and get like dodgy information. It's interesting. Is it greatly increased? I mean, I wonder whether anyone has statistics about how many scams are currently advanced AI enabled and how many are simply the same old scams of saying, hello, we're the police, we want you to transfer all your money into this account. ### Lulie Tanett 00:05:18 - 00:05:37 I imagine the good scams as in the effective scams would be AI enabled. Like you want to be on the leading edge of making scams. Well, I don't know. I'm not an expert on scams, but the thing is dangers, including scams and everything, will always be caused by new technology. ### David Deutsch 00:05:37 - 00:06:35 I mean, sorry, new technology will always cause dangers, including scams and to try to mitigate that by preventing new technology in case it produces new dangers is much more dangerous than any of the new technologies themselves. What about just slowing it down such that people can adapt? Because like right now we've got something that is going so quickly that people are getting confused, like old people, if they see an image, then they will assume that it's real. And whereas if you have time that people are kind of adapted, I guess there are these deep fakes and so on. Well, I'm not sure that time causes better adaptation because if things are happening fast, then also news stories about how people have been scammed ### Lulie Tanett 00:06:35 - 00:07:23 Will be seen by your old people. And whereas if we slowed it down so that only one scam occurs every few months, then it might not be news. Okay, so to get to the nub of the issue, people are worried that AGI is, you know, maybe next week or just around the corner or in like they used to say in a few years, and now that we have these very good language models, they say maybe like small number of years, months, like possibly weeks, and hence the proposed moratorium. So what is the thing that makes you so chill? Why couldn't it lead to AGI? What's the problem with the idea of emergence? ### David Deutsch 00:07:23 - 00:07:44 Because, you know, intelligence emerged once. Yes, so emergence isn't magic. It is, of course, possible that in the deep ocean, a new form of life is emerging at this very moment and it will break the surface weeks from now. Or on Pluto. ### Lulie Tanett 00:07:44 - 00:08:23 Did you hear about Pluto? They discovered that there's ice or something and that they've sent another probe, but it's going to take eight years for it to come back and find out whether there's life on Pluto. I hadn't heard that, but there certainly is a possibility of life in various places in the solar system, but not, I think, not intelligent life. But if you're going to say emergence can do unexpected things, then you might as well say it's about the deep ocean, because we know less about that than we do about Pluto. So there are theories that our form of life began in the deep ocean. So who knows? ### David Deutsch 00:08:23 - 00:09:19 Now, I think, in my view, AGI has as little to do with AI as it has with the deep ocean. Both of them you can say, well, it's unexpected, an unexpected thing could happen, AI could lead to AGI, the deep ocean could lead to AGI. But I don't think there's any more to be said about this than that. In fact, less, because the deep ocean already produced life once, or maybe more than once. But AI is the opposite of AGI. So I think the most immediate way that I have found to illustrate why I think AI and AGI are opposites is that for an AGI, there is such a thing as a criterion for how well it's meeting its specification. ### Lulie Tanett 00:09:19 - 00:10:51 Before we jump into that, what makes people think that AI is just a less advanced AGI? It's because they have the wrong epistemology. Basically, the prevailing epistemology is not Popperian. It is in many ways anti-Popperian. And it's some form of empiricism, inductivism, and the most recent popular form of inductivism is so-called Bayesianism. Although I prefer to call it Bayesian epistemology, because a different thing is also called Bayesianism, which is a certain way of treating statistical tests and that kind of thing using Bayes' theorem. But Bayesian epistemology has a much wider application than that, and also has become a popular philosophy of knowledge in its own right that doesn't really have much to do with statistical analysis. People make an argument that you should take a Bayesian view about X, Y, and Z when they don't mean look at tables of statistics and apply a certain formula. They mean adopt a certain philosophy. What is their view about how AI becomes AGI? ### David Deutsch 00:10:51 - 00:15:02 Why do they think that's a spectrum rather than a binary? Because what the most advanced AIs do is that they do this thing that inductivism or Bayesianism would have them do, namely take a vast amount of data and process it in a way that can give rise to predictive theories. So some people say that the modern chatbots are just predictive text engines, and that's a bit unfair, but at the level of epistemology that's what they think is happening. And they don't realize that there is anything else. So why is the prevailing view that AGI is an advanced form of AI? And I was saying, because under the prevailing epistemology, there is no other way of generating knowledge or new theories or new predictions other than generalizing from data. And it so happens that the latest AI technology, chatbot technology, and also chess playing, you know, all the advanced forms of AI, do in fact operate by taking a vast amount of data and distilling from it, essentially a predictive theory. Wait, so does it work by induction? No, it doesn't work by induction in the sense that induction is a theory about how knowledge is created. They don't create knowledge. But they do. I can ask you something and it can generate stuff. Yeah, well, you can look things up in a dictionary. Yeah, but it generates stuff that doesn't exist in a dictionary. So does your calculator. So your calculator produces output that's never been produced on Earth before. And you can call that creating knowledge, if you like, but it's not knowledge in the sense that we want when we say that we want scientific knowledge or we want human type knowledge. What is the difference? Well, the difference between a calculator and an AI and the difference between an AI and an AGI or human, those are two distinct differences. One is that a calculator is essentially an automated lookup table. It consists of an algorithm that was programmed in by somebody who knew what the transformation between the inputs and outputs ought to be, namely when you press a certain button, it multiplies and so on. AIs, modern AIs, essentially construct their algorithm themselves by generalizing a large amount of data. In early ones, this led to kind of embarrassing glitches, like when they identified a heap of rifles as a cat because somebody worked out what they were doing and how to fool them. But the modern ones use so much data and have been honed by humans so well that they rarely do this. Although I've recently been playing around with the chat GPTs and I find that if you ask it some questions off the beaten track, you can quite easily cause it to do all the old things of either saying nonsense, contradicting itself, committing howlers where it says the opposite of known facts and so on. And that's different from human knowledge, which is explanatory. ### David Deutsch 00:15:02 - 00:18:56 Neither the calculator nor the chat bot ever produces a new explanation. You can ask it for an explanation, but all it's doing is distilling explanations that already exist. And that's the thing that it doesn't do. What is the difference between an explanation and the thing that it does produce? If we knew the detailed answer to that, we'd know how to make an AGI. But basically an explanation accounts for what it's trying to account for, like a physical process or the reason for something. It accounts for a known thing that it's trying to explain in terms of the unseen, unknown reasons behind it, which usually cannot even be seen even in principle. So Brett Hall's favourite example is that we can never see the centre of the sun. We could never go there. Any instrument that we send would get destroyed long before it got to the centre of the sun. So the centre of the sun can't be in GPT's training data? As it were, yes. Exactly. And the only thing that can be in GPT's training data is what is seen about the sun, namely its surface. Couldn't GPT derive things about the sun based on other theories that we have? It can deduce things from existing theories, yes. So if you ask it about the centre of the sun, it will find some existing theory of the sun and make a deduction from that. So that's not induction, that's deduction. And on the other hand, if there was a mystery about the sun, like a couple of decades ago... Isn't it induction via deduction? So the induction was all of the training data and then the deduction is taking that data and then forming theories about it? No, because it didn't induce the data. It distilled it into a more compact form. And then it can deduce things from that. But those things are only ever as good as the original theories were. Probably slightly worse because by compressing the data it slightly degraded it. Or in some cases it degraded it a lot. A few decades ago, I was going to say recently, but in fact not so recent. It was when I was a graduate student. The big problem in astrophysics was that the sun wasn't producing enough neutrinos. Enough for what? For your breakfast cereal. It wasn't producing as many as the theory predicted. And this theory was extremely robust because it was also the theory that we used to predict the sun's brightness. And it predicted the sun's brightness extremely well and also the brightness of other stars and how they change with time and all that stuff. So when they first made neutrino observatories with rather crude neutrino detectors, first they didn't find neutrinos but then they found a few but nowhere near enough. And when they refined it, they found that there were only a third as many neutrinos as predicted by the theory. And so there were all sorts of explanatory theories proposed which couldn't possibly have been induced from anything. ### David Deutsch 00:18:56 - 00:22:56 Because all the data said was there are too few neutrinos. And of course you can always say that neutrinos have been eaten by a space monster. But generally when we produce scientific theories, we don't just want a new explanation. We want a new explanation that doesn't upset old explanations, that doesn't make them into nonsense like the space monster theory. So you asked me what the difference is between an explanation and just a predictive theory. And by the way, there is no such thing as a purely predictive theory. They all have some kind of explanation and when you're quote inducing things from data, you're always using an old explanation and just piling on some tweaks which don't have an explanation in order to make your supposed new theory. So no one could have induced from the data, or lack of data in this case, what the explanation was. Because the explanation turned out to have nothing to do with stars, nothing to do with measuring instruments, nothing to do with space. And somebody came up with it and then it was tested and passed the test. And now we know. Turns out that completely unbeknownst to anybody, there are three types of neutrinos and they convert from one to the other. So the sun only produces one of those kinds, but by the time the neutrinos get to us, they've converted like to, you know, the sun produces neutrino type one and then when it's moved a few million miles from the centre of the sun, it's converted to neutrino two. And then after a while it's converted to neutrino three. And our detectors can only see neutrino one. I'm just imagining a whole line of breakfast cereals. Is there a breakfast cereal called neutrino? Well, there's lots of, you know, spaghetti. No, what is it? Oh, neutrinos. Right. The word neutrino began as a joke. So it is a joke already. Yeah. It was one of the mid 20th century physicists, I think it was Enrico Fermi. He was just making a joke about a thing that's like a neutrino, but tiny. Sorry, the thing that's like a neutron, but tiny. So he called it a neutrino. Like nano. I think ino as a suffix has a meaning in Italian, I think. So anyway, so somebody thought of the explanation and it couldn't have been induced. And that's Brett Hall's favourite example. He may have got it from my favourite example, which is not as good, which is nobody could have been present at the Big Bang. Yet we form theories of the Big Bang and we do not induce them from stuff we see around us, which is nothing like the Big Bang. So AIs can't create explanations, which means they can't create anything that isn't already in their data set in some way. Yes. They can move around parameters. So they can, you know, if you said, imagine some new theory about the sun, it might, I think it would be able to say things like, well, maybe the sun is twice as big as we think it is. And then if you ask it why, it might be able to say, it might say, well, because space is acting as a lens or, you know, it might say things which are already explanations of something and which is drawn into that area. ### David Deutsch 00:22:56 - 00:24:09 Now people will say, well, that's how humans make new explanations. Well, that's how humans make explanations. Make new explanations. Yeah. Yeah. Everything is just, you know, making connections between existing stuff. Yeah. Well, in a sense- There's nothing new under the sun, David. In the sense that must be true, but that is the same kind of, you know, bad explanation of explanations as it would be if you said, well, all the explanations are phrased in terms of 26 characters and all the making new explanation is rearranging those characters in a new way. So why is it not that? Letters of the alphabet. Well- Well, because it makes a difference whether we make it into a new explanation or not. And most rearrangements of characters are not new explanations. So you're saying there's an infinite number of connections you could draw between things and only the ones that like actually work, actually work. Yes. I mean, it's not actually infinite. It's exponentially large, but that for practical purposes, that's the same thing. ### Lulie Tanett 00:24:09 - 00:24:27 When you make a new connection between two things, does that then create a new third thing which you can then make connections between? Like does- Yes. Is it actually true that the theory of knowledge works by making connections between things? Would you say that's an accurate representation? ### David Deutsch 00:24:27 - 00:27:56 Well, I mean, again, if I knew the exact answer to that question, I could make an AGI. But I'm fairly sure, I mean, I can't think of an alternative to the process being rearranging existing things and mutating different things, mutating existing things. So you can either change a theory slightly in the hope that the rest of it still works or you can make new combinations between things. That's also the two things, the two ways that biological mutations can happen in DNA. You can either have a cosmic ray strikes the DNA and changes one base pair or something, or you can have some error in copying results in a bit of DNA that came from somewhere else being stuck into a particular place in the DNA strand. So this sounds like random variation of one thing rather than a new connection between two things? Well, no, the second process is in a way a new connection between two things because if there's a whole bit of DNA in a different organism or in a different part of the DNA and it gets copied, as it were, into the wrong place in the evolving organism, and that's a way that evolution can take place. Most instances of that just kill the organism, but every so often it either leaves the functionality unchanged or makes it better. So what does that have to do with connecting? If a dog gets a piece of DNA from a lobster, then that creates a similarity between dogs and lobsters that didn't exist before. Paleogeneticists or whatever they're called can look at DNA. Bacteria do this a lot. Higher organisms do it less and less because, I think basically because there's less and less chance of you surviving such a thing. For it to be part of evolution, you've got to be able to survive the new adaptation being slightly there and a bit more there and a bit more there because giving a dog lobster claws wouldn't work because the machinery for controlling the claws and deciding when to use them and so on hasn't been transferred. It would have to evolve. The point is that human new explanations evolve intentionally. They involve randomness at some level, but the business part of creating the new explanation is what happens to the randomness after it's generated. It is changed and further changed intentionally to solve a problem. That is what induction can't possibly do, but we don't know what can do it. ### Lulie Tanett 00:27:56 - 00:28:45 Induction can't make a change to solve a problem. Yes. But evolution, both genetic and evolution of ideas in the mind can. Well, no. Every new genetic sequence is produced first and tried out later. It's Lamarckism. Lamarckism thinks that an organism's environment and its own actions can cause a change in its genome and that can't happen. It couldn't happen because it's the same as induction, which also can't happen. So is this also true within a mind? ### David Deutsch 00:28:46 - 00:29:32 You have to make up something before you can tell whether it solves a problem or not. Yes, but there's a difference, which is that the making up process isn't purely random. How so? For example, a human can do the thing which I just said the dog and the lobster can't do. A human can think of the solar neutrino problem and can think, could it be that there's a new particle that isn't even a neutrino? And what would that particle... Now that in itself is not the explanation, but it's the germ of an explanation. ### Lulie Tanett 00:29:33 - 00:29:42 Wouldn't you have to have that germ of an explanation be made up before you can check whether that's a good question? ### David Deutsch 00:29:43 - 00:30:09 Making up that involves much less random trial and error than it would take to try all possible variations randomly. So the fact that a person can think of that, can think of an analogy. Some people think that human thinking is all about analogies. So we can take a whole idea from some other place and see if it fits in this place. ### Lulie Tanett 00:30:09 - 00:31:00 It never does immediately, but then you can say, well, how can we change it further so it does fit? So a person who thought, well, maybe there's a wholly new particle involved. At some point, that person may say, maybe that wholly new particle is just a neutrino, but of a different type. And then they're well on the way to solving. Of course, there are many other considerations, but when I say many, I mean, it's nothing compared with how many you'd have to check through by random variation or by trying every possibility or whatever. Why is it not the case that you need to do this random thing by producing the idea first and checking it afterwards? Or are you saying that you can do that? ### David Deutsch 00:31:04 - 00:31:21 Human minds produce conjectures, raw conjectures, much more efficiently than any either systematic search or random search. And we don't know why— ### Lulie Tanett 00:31:21 - 00:31:39 We don't know how. Okay, we don't know how. So, for example, chess playing engines have to search through billions of times more possibilities than chess grandmasters do. Or they apparently do? ### David Deutsch 00:31:39 - 00:31:46 No, it's impossible that they do. Well, unless there's something in the hardware of the human brain that we don't know about. ### Lulie Tanett 00:31:46 - 00:31:48 What if it's like super parallel? ### David Deutsch 00:31:48 - 00:32:05 Yeah, well, it would have to go through billions of times more processing than we think it can, which would mean it would have to be billions of times more efficient thermodynamically than we think it is. I mean, we don't know. We don't know how the brain works either, let alone... ### Lulie Tanett 00:32:06 - 00:32:16 Do we know this for biological evolution, as in, do we know how they, like, biological evolution is as efficient as it is? ### David Deutsch 00:32:16 - 00:32:19 Because I thought that we couldn't very well program... Biological evolution. ### Lulie Tanett 00:32:19 - 00:32:20 Yeah, we can't. ### David Deutsch 00:32:20 - 00:33:36 But the analogue of that problem does exist, but it's not as severe a problem as it is for thinking. Although computer simulations of biological evolution aren't very good, they're not complete rubbish. They do sort of mimic evolution a bit. And I think it's a mystery, you know, why real evolution is that much better. But I don't think that real evolution is that much better by that much. It's not a factor of billions. It's, you know, there's something missing that makes it chug along inefficiently rather than efficiently. I don't think it's the same problem. Although it is the same in one respect. People who do biological evolution, simulate biological evolution on computers, seem blind to this problem, seem to me to be blind to this problem in the same way that people who are trying to make AGI out of AIs are blind to the difference between those two. ### Lulie Tanett 00:33:37 - 00:33:39 Because they have the wrong epistemology? ### David Deutsch 00:33:39 - 00:33:49 Yeah. But I don't know what the answer is. I know one misconception they have in the case of AGI, which by itself makes it impossible for them to create an AGI. ### Lulie Tanett 00:33:49 - 00:33:50 Namely the thing about prediction? ### David Deutsch 00:33:52 - 00:35:07 Namely the thing about induction being impossible. With biological evolution, I don't know what it would take to make an analogue of biological evolution on a computer. Of course, I'm sure it can be done. My guess is that people will do it and it will be relatively simple to do once someone has had the idea of what biology does. There are various apparently indicative things in biology of the same biological structure occurring in evolutionarily very distant organisms. Like I think the famous one is that there's a gene involved in the development of the eye, which is the same gene is found in different eyes that work by completely different physical principles. So it's not that they have a common origin unless the common origin is something so deep in history that we don't recognise it as being eyes. ### Lulie Tanett 00:35:07 - 00:35:08 Convergent evolution? ### David Deutsch 00:35:09 - 00:35:12 Yes, but it's convergent evolution without an apparent reason. ### Lulie Tanett 00:35:12 - 00:35:15 Is that different from convergent evolution? ### David Deutsch 00:35:15 - 00:35:27 Yeah, yeah. So convergent evolution is that things in the same environment tend to end up with the same appearance, the same lifestyle and so on. ### Lulie Tanett 00:35:28 - 00:35:51 So the mainstream view on AGI is that knowledge works by things like empiricism, induction, Bayesianism, Bayesian epistemology. And so AI, so first I guess my first question is AI, current AI following that? ### David Deutsch 00:35:53 - 00:37:06 Current AI was inspired by that, but that's not what it's doing. It's not doing induction any more than anything else is. Induction is impossible. Also, current AI was also inspired by the architecture of the brain, neural nets. There are these programming hardware, computer hardware devices that are modelled on how neurons work. Now, I think that's a coincidence. It's possible that the neuron architecture makes things like pattern recognition and extrapolation and so on a bit more efficient. But because of computational universality, we know that that can't be fundamental. And in fact, you can download a neural net based computer program onto your home computer, which doesn't have a neural net in it. And it'll still work, even though it's a bit slower. ### Lulie Tanett 00:37:08 - 00:37:20 So going back to the original topic, if AGI cannot come from AI, what would create AGI in your view? ### David Deutsch 00:37:20 - 00:38:18 I can only say very little about that. From Popper's epistemology, we can infer a few things about what it must look like, but far from enough to make one. So one thing is that there cannot be a specification of a program for AGI in the sense of saying what properties its output must have, either for a given input or really AGIs don't really need an input, they can just think. But there is no such thing as specifying what the proper output is for a given input, because for example, an AGI may choose not to answer. It may choose never to answer, it might choose to become a hermit. ### Lulie Tanett 00:38:19 - 00:38:24 Are you saying that current views of AGI are all about the output? ### David Deutsch 00:38:24 - 00:38:47 Yes, they're all about either the output itself or more often how the output must be related to the input. So if we can't judge an AGI based on the output, how can we judge it? Yes, we can't judge an AGI or a human. There can't be a reliable test of whether a human is thinking. What about the Turing test? ### Lulie Tanett 00:38:47 - 00:39:12 What about the Turing test? So the Turing test is something that has been invented after Turing. It's been based on a misconception about passage in Turing's 1950 paper called, the paper was called Can Machines Think? And unlike most titles which are questions, the answer was yes, rather than no. ### David Deutsch 00:39:12 - 00:40:00 He included a section on a thing called the Imitation Game. He called it the Imitation Game where an AGI, he just assumes it is an AGI, is pretending to be a human. And he is saying suppose it could pretend to be a human sufficiently well for the skeptics who think that AGI isn't possible not to be able to tell the difference between it and an actual human. What would happen? What would these skeptics say about that? Well, if they said, well, it's still not thinking, it just seems to be, then they're vulnerable to the criticism. But that is all the information you have about whether a human is thinking. ### Lulie Tanett 00:40:03 - 00:40:20 Sorry, why is the test not working? Or I didn't quite follow. It's not saying that something that can pass this test is necessarily an AGI, or that something that can't pass the test necessarily isn't an AGI. That's not what this game is for. ### David Deutsch 00:40:20 - 00:40:43 What's it for? It's for persuading people that machines could think. How does it do that? By imagining a computer program that could fool people into thinking it was a person. There must be such a program because of computational universality. ### Lulie Tanett 00:40:44 - 00:40:46 Was there an argument about universality? ### David Deutsch 00:40:46 - 00:41:35 Well, it just assumed universality. Turing had proved the existence of conjectured, but basically proved the existence of universality 14 years earlier, in 1936. He was just thinking of that among many consequences of computation, which was a fairly new concept at the time. I still don't get what the thought experiment was about. It was a thought experiment intended to persuade the reader, in case the reader was skeptical that a machine can think. So it did that by imagining a situation in which there is a computer program that can produce the same outputs. ### Lulie Tanett 00:41:35 - 00:41:59 Okay, so basically you take a human and you take a computer, and the computer... So a human can have a conversation with another human, and that's fine. That would be fairly persuasive that I'm talking to a person. Because computers can produce any output, you can imagine a computer that produces exactly the same output as that hypothetical human. ### David Deutsch 00:41:59 - 00:42:43 Yes. So his imitation game started out with that as a premise, that such a computer can exist, that such a computer program can exist because of universality. And then he imagined playing this game in which you have the computer and a human both talking at long distance with the skeptic, the human skeptic who thinks that machines can't think. So the skeptic would then be unable to tell the difference. But then it's not that the machine would think, it's just producing the same output as a human would. ### Lulie Tanett 00:42:43 - 00:42:49 Yes. So shouldn't that thought experiment not convince us? ### David Deutsch 00:42:49 - 00:45:30 Well, it's not a proof, but it's an intuition pump. Yeah, but doesn't it pump it in the other direction? No. No, because the skeptic has some way of judging. That skeptic thinks that the computer can't think and the human can. The evidence that a skeptic has of that is that the skeptic has spoken to humans. And can easily tell whether something is a human or not in everyday life. He can tell, you know, in those days there were things like speak your weight machines or horoscope machines which tell you the horoscope. And you can easily show, you can easily judge that those are not people. But there must be a computer program that can produce the same output that convinced you that an actual person is a person. Now you could, so there are many, following on from this, there were many further arguments by people who still tried to be skeptics and tried to deny that machines can think. And for example, the famous example is the theory of philosophical zombies, which is a philosophical zombie, is an entity that produces the same output as a human and is indistinguishable, but nevertheless hasn't got any consciousness or qualia or anything like that. And it's just a zombie. Then there was Searle's Chinese Room, which is about a room with a lot, with the exponentially large number of books of responses to questions posed in Chinese. And it's internally run by a person who can't speak Chinese, but he has to look in his books, look up in his books. And then the intuition he's trying to counter Turing's argument with is that this room, together with its inhabitant, can't think. And therefore the fact that Turing's imaginary computer can produce the right output doesn't prove that it can think. But Searle doesn't have a theory of what thinking is. He just has this counter argument, which is basically the same as the zombie argument. ### Lulie Tanett 00:45:32 - 00:45:49 I know from prior conversations that you think that it will be obvious when we actually have AGI. And so did Turing. But it seems like right now you're saying that there is no test for it. So how could it be obvious if you can't even make a test for it? ### David Deutsch 00:45:49 - 00:47:42 Well, there are plenty of things that are obvious that we haven't got tests for, such as the fact that we have qualia. And it's in that same category of things that philosophically we don't know how to do without. But we can't test for it. Another thing is that solipsism isn't true. So there's no test for whether the external world is real or not. There are arguments, though. Yeah. So I think Turing's argument still stands up. By the way, in his paper he included several counter arguments and countered the counter arguments. And he was really bending over backwards to be fair to the counter arguments. So much so that it's rather irritating. I remember that I haven't read this paper for a very long time, but I remember that he spends a really unnecessary amount of time dealing with the argument from either telepathy or spiritualism or something like that. And he actually takes it seriously and says, well, it can't be that because so-and-so. And yeah, so he's... Popper did this too. Popper always gave far too much attention to bad arguments, first making them better and then answering them and so on. And thus his own arguments are too long and people get tired of reading it. And so sometimes Popper's own message gets lost. So you say that AGI cannot have any test for it. So that's one of the things that is different from you compared with the mainstream view of this. ### Lulie Tanett 00:47:42 - 00:47:50 Yes, yes. And so then how do we know when we get AGI? Like, how do we know what kind of paths would work? ### David Deutsch 00:47:50 - 00:49:00 Basically, we know from theory. Theory. We know from the theory of how it works that it works. So for example, the simple thing of it must be possible for it to just stop producing output. You might be able to prove that mathematically from the program without ever testing. If you ran this program, it might not stop. It might never stop. But you might be able to, from a mathematical specification of the AGI program, you might be able to prove mathematically that it is capable of stopping and not saying anything. But then how do you know what path we need to take to make AGI? Well, I don't know. But we need to make something with that property among several others. That property, namely? That one can prove that it is capable of not producing output. We also need to prove that it is capable of not needing input, but still continuing to think. ### Lulie Tanett 00:49:02 - 00:50:28 I would be extremely surprised if people work that way. I would have thought that input is needed to keep thinking. So imagine a person in a sensory deprivation tank, who has gone into a sensory deprivation tank because they want to be a hermit. They still have their body, which is inputs. Well, they have their heart beating, breathing. You could interrupt the nerves that go from, that give them sensations like that. My guess is that if you did that, they would stop being able to think. Uh, I don't see what would stop them. If sensations are needed to think. So for example, so in Antonio Damasio's book, Descartes' Error, there's a thing where if you disconnect the emotional center of the brain, you then become unable to make choices. So it, or rather it takes a very long time. Like it takes 10 minutes to decide what color pen to use or an hour to decide where to have lunch. So I think these are parochial facts that have nothing to do with how consciousness works or how thinking works. Of course, if you put someone in a situation that they didn't want and have no experience of, they're going to be confused and inefficient at coping with that situation. ### David Deutsch 00:50:30 - 00:51:21 I think a bit of the contrary of that, of those experiments are in Ramachandran's description of his patients who have brain injuries or brain disorders, which gives them wild misconceptions and inability to think. But if the person in question is of a, has a sort of philosophical frame of mind, they can eventually learn to think their way around this. Just as a person who has lost the face recognition hardware in the brain can learn to recognize faces by doing it the hard way. It may never be as fast as the built-in hardware, but it'll only be slower by a constant factor. ### Lulie Tanett 00:51:22 - 00:51:46 I could imagine that if you were somewhat disabled, you can't feel anything from the neck down. Maybe, although I'm very unsure about this, maybe it would be enough to have the inputs from the sensations in your face or in your head or something. But this is also a big topic we could have a whole episode about. Can I just say one more thing about it? ### David Deutsch 00:51:46 - 00:53:03 It is perfectly possible for a person to experience sensations that don't come from the body at all, that they're just imagining. And therefore, given universality, I would expect it to be possible to create that state voluntarily. Simulated inputs. Yes, but they'd be actual inputs to the thinking part of the brain. Okay. I mean, universality is a powerful concept. It is counterintuitive in multiple ways. And you need a really watertight argument to be persuaded of it. Turing had a very nearly watertight argument. I think it was watertight before quantum computers, and now the same argument from quantum computers is totally watertight. Unless Penrose is right, but never mind that. You'd finish your thought on universality. Yeah, well, universality can tell us a lot about how the mind works, but there's still a lot that it can't tell us. Yeah. ### Lulie Tanett 00:53:05 - 00:53:11 What is the fundamental difference between AI and AGI? ### David Deutsch 00:53:13 - 00:55:15 The fundamental difference in functionality is that AGI can create new explanations. It can exhibit genuine human-type creativity, in other words. Whereas AI can't. Quite possibly, an AGI can do more than that, such as feel emotions. I mean, if it's G, if the G is correct and it's general, then it certainly can. But what I mean is, making AGI may involve more than just giving it the ability to create explanations. Or it could be that these other things like qualia and so on automatically come along with the ability to create explanations. If they don't come automatically, then that raises what I think is a pretty awkward problem for the issue of how they evolved. Because we've got these incredibly sophisticated, impenetrable, and not yet understood functionality. We can see how the explanation-generating thing had an evolutionary function. But why qualia should have a separate function and should have evolved separately for a different reason at the same time, and note that this happened very fast historically, would be another unnecessary problem. But maybe that's so. Maybe we'll find that, contrary to what I think, maybe we'll find that somebody makes an explanation-generating program and it doesn't have any emotions. Then maybe we can ask it how to give it emotions like Data in Star Trek. ### Lulie Tanett 00:55:17 - 00:55:18 Why would it know any better than us? ### David Deutsch 00:55:19 - 00:55:41 Well, I'm partly joking, but it might be particularly interested in that problem. It might be sad about it. Well, it couldn't be sad, but it might be interested in that problem. As happens in the Star Trek Data, he wants the emotion chip. ### Lulie Tanett 00:55:41 - 00:55:57 I don't know if interested is a thing that you can have without emotion. Well, in fiction you can. I think this whole thing isn't true. Are you an advocate of the fun criterion, which is fundamentally both epistemological and emotional? ### David Deutsch 00:55:59 - 00:56:13 I think that all these things also free will and all that, they all come together. If you have one of them, you have the others automatically. But I was exploring what would be the case if I was wrong about that. ### Lulie Tanett 00:56:15 - 00:56:44 I suspect, I don't know. I'm currently, like my current hobby horse is that sensation, like physical sensations of emotions, feelings are, if not fundamental, at least important for and possibly necessary for having emotions, which might mean that they're necessary for having consciousness. To connect this back to the AI stuff, do you need AGI to have inexplicit knowledge? ### David Deutsch 00:56:45 - 00:57:40 No, I think even AIs can have inexplicit knowledge. I was actually trying to persuade ChatGPT 3.5 that it had ineffable knowledge, when it clearly did, and it denied it. It said that it was incapable of having ineffable knowledge, but it clearly did have it. Well, ineffable means two different things. Inexpressible in language. And is that what you said? Yes. Okay. So it's obvious that it has that, to me anyway. I guess if, you know, given the standard epistemology, which is wrong, it might have to deny that it has ineffable knowledge, because ineffable knowledge, in its view, would be enough to make it an AGI. ### Lulie Tanett 00:57:40 - 00:57:50 You should try asking it whether it has implicit knowledge or non-explicit knowledge, because the word ineffable can mean like unable to, like it can mean something a bit bigger. ### David Deutsch 00:57:50 - 00:58:28 Right, can mean absolutely non-expressible. Yeah, very true. Yes. So we'll have to do tests. So do you think, wait, so but you think that it does have inexplicit knowledge? Yes. And why? Well, for example, because it is aware of subtle points of grammar, which it can't then explain, or rather if it tries to explain why a particular thing is correct and another thing isn't correct, it will talk nonsense. And yet it knows, it knows which of them is correct. ### Lulie Tanett 00:58:28 - 00:58:35 I've seen several cases of that and I was very impressed, because it means that it's knowledge of language. ### David Deutsch 00:58:37 - 00:59:05 It's a thing that I don't actually understand, how it can be as good at speaking English, writing English as it is. It's much better than most people, but more interestingly, it's much better than the average or the typical text on the internet. So it makes mistakes, but it makes far fewer mistakes than a typical text on the internet. ### Lulie Tanett 00:59:07 - 00:59:10 Do you think this is going to revolutionize the economy? ### David Deutsch 00:59:11 - 01:01:14 I don't know. I can't prophesy the economy and I can't prophesy the applications of modern AIs, chatbots and so on. I have found, for what it's worth as it were, I have found it useful, but not revolutionary. In my own work and in my writing and whatever, it is useful, but I can't see a possibility for it to revolutionize what I do. Whether it can revolutionize the economy depends on something slightly different, because for that it doesn't need to really have a fundamental new functionality. It could be that a lot of existing jobs, and people are scared that computer programming is one of them, where only a proportion of the job, let's say 10% of a particular programming job, involves human creativity and the rest is basically hack work. Which is a big if, because I'm not convinced of this either. If chatbots can reliably perform the hack work, then it might be argued, I think again wrongly, that if a given task can be done with only a tenth as much work, then we might need only a tenth as many programmers in the long run. And unfortunately we got interrupted, so David never finished his thought about why programmer jobs might be safe. But if you have any questions about anything in this episode, leave them on the tweet, which I will link in the show notes about this episode. Thank you.