2021-08-29 Eli Tyre Popper's Problem-Oriented Epistemology with David Deutsch and Eli Tyre
Duration: 01:31:42
Transcript
Eli Tyre
You’re kind of taking on board Popper’s idea that ideas don’t matter where they come from. And what matters is what happens to them. So what happens to them in your picture is that they are subjected to criteria. Specifically ideas that I would call goals. I think there are more kinds of critique that are available to other sorts of ideas. Like for instance, yeah, if I’m evaluating a scientific theory, I guess maybe I could still reformulate that as criteria. But I’m going to compare it to different kinds of ideas depending on the sort of idea that I’m thinking about. Like the sort of idea of like should I go to the gym today or should I walk to the grocery store and get a chocolate bar? Has some different considerations as to whether or not I’m going to accept it even tentatively than asking the question like do I think there’s a tree outside or is it raining or what is the fundamental nature of physics?
David Deutsch
So all those things you’ve mentioned, none of them are problems. None of them take the form of problems which according to Popper is where thinking actually begins.
Eli Tyre
A problem is a conflict between ideas?
David Deutsch
Yeah, that’s an aspect of problems that I’ve mostly stressed. I’m not sure that Popper would put it that way. He might want to say that many problems are the form of conflicts between ideas, but many aren’t and he would say that even an amoeba has problems, but it doesn’t have any ideas. And still I think the conflict between ideas idea is still there even in those cases, even in amoeba, the problem is the implicit problem of surviving in an environment and the conflicting ideas are basically the existing genome and a mutant. And until you have an existing genome and a mutant, there’s no evolution. There’s no natural selection. Nothing can happen. And what’s more, well, nothing can happen until the solution has already been guessed. So again, it’s important in evolutionary biology. What’s in common with the amoeba and the person who’s going to go down the road to fetch a hamburger or whatever it was? The thing is that the idea comes first. It doesn’t come from wondering what I should do unless that’s a problem to you, unless what I should do is a place in your mind where there are conflicting ideas and that problem has come to the top of your scheduling ideas. Normally when we go to get a hamburger, when we put one foot in front of the other, we’re not wondering where should I put my foot? Where should I put my foot next? We’re not doing anything like that. We just have an existing idea. And since there’s no problem, there’s no thinking.
Eli Tyre
My guess, so I’m not sure if this is a tangent. My guess is when you are literally walking, there is in fact a bunch of little bits of conflict where you have sense data, although I think we may need to clarify what sense data means. I’m fine saying sense data are ideas, although also theory-laden. You put your foot down and you have a little bit of sense of being off balance or on balance, and there’s a bunch of low-level processes that manage this such that something goes a little bit wrong and you correct it.
David Deutsch
There are things happening in the human body like the immune system, which is also reacting to things, doing one thing, stabilizing itself, doing another thing, changing levels of importance and so on. None of that is thinking. None of that is creating knowledge. There are things that the brain does also that aren’t thinking. They are just automatism. When I wonder what three times five is, what I’m actually doing is just retrieving a memory. If I accidentally retrieve, if there’s some corruption, this memory was formed by rote learning somewhere sometime in my childhood when something was messed up in my brain. Therefore, the only thing I can think of when someone says, well, seven times three, seven times three, I think of 21. If I accidentally think 31, then it might be that an error correction process comes in. What it’s doing, we can anthropomorphize it by saying, oh, the 31 sounds a bit wrong. Let’s go back to try to retrieve our memory and do some error correcting on the memory, and then let’s get back to 21. That’s all things that happen in the brain, but they are not creative. They’re not creative thought. Similarly, there’s lots of things that an amoeba does. Most of the things that an amoeba does aren’t part of evolution. Any of them can be part of evolution, but mostly what an amoeba does is enact its existing theories, which have within them algorithms.
Eli Tyre
Let me see if I can summarize that. Let’s start with human bodies. In human bodies, there are a bunch of systems that I would say are epistemic, although maybe that is too strong. The immune system in particular does something that looks like learning. It will encounter some stimulus, and it will store… Yeah, I guess you wouldn’t say it creates knowledge. Actually, I’m not sure if you would say it creates knowledge. It sounds like not. It definitely doesn’t. But it does do something like store memories.
David Deutsch
Yes, so does a single bit of computer memory. Yeah, okay. And if it doesn’t learn. Okay, that seems right.
Eli Tyre
So there are many processes that store knowledge in various ways or reflect the state. You can write an algorithm that will take in some input and produce some output, and some of that output might be, and now store this in memory. I have in fact written algorithms that do that. You might also think of a sunflower, which is apparently responding to the world. Like the face follows the sun. But again, if you look at the details of it, there’s a specific process. The side that’s in the shade grows a little bit faster, and so it tends to lean towards the sun.
David Deutsch
And that algorithm that the sunflower has, that does contain knowledge. But what the sunflower itself is doing is not generating any knowledge. That knowledge was generated over a million years by natural selection. That knowledge is already there, and no new knowledge will be created until there’s a problem.
Eli Tyre
Okay, give me a moment to think about that. No new knowledge will be created until there’s a problem.
David Deutsch
And probably not even then, but it’s only very likely. All right. When the combine harvester comes, it may cause a problem in some sense. But usually it’ll just result in death. But the sunflower’s knowledge arose from those few cases where some ancestor of the sunflower survived death. Yeah. Right.
Eli Tyre
So I definitely don’t think that the sunflower is accumulating knowledge. I do think that there is knowledge embodied in its structure.
David Deutsch
Yes. Right.
Eli Tyre
But humans only acquire knowledge when they encounter a problem. What about just playing out logic? Like I start with a set of premises, and there is implicit in those premises a set of conclusions. I might not start out knowing, although maybe you would disagree with that. I might start out knowing the conclusions of some logical premises. But if I sit down and think and write it out on a piece of paper, I’m like, ah, this mathematical proof is actually kind of surprising to me. But it turns out this is the case. I knew I had all of the pieces all along to derive that. But I actually needed to sit down and derive it. And at the end, it seems like I have new knowledge.
David Deutsch
It’s a good example, because I also think that isn’t knowledge either. But it looks much more like a simulacrum of knowledge than an amoeba or a sunflower. So what happens much more often in regard to mathematical proofs is that somebody can learn a proof by heart, can verify each step according to rules of inference, and then doesn’t understand it at all.
Eli Tyre
Yes, I’m also familiar with this phenomenon.
David Deutsch
So all this goes to show, I think, that proving is not the same as understanding. And following an algorithm is not the same as creating knowledge.
Eli Tyre
Okay, so I agree that proving is not the same as understanding. And it seems like the thing that you’re saying is, in order to have understanding from a proof, there’s also resolving problems, which does relate to, like, that does reflect my experience of doing math. Like, I will sit down and I sort of have a bunch of intuitions that I’m working through. I’m like, ah, it sort of seems like this, but okay. But what about with this thing? I sort of push those two ideas together until I’m like, ah, okay. Yeah, all right. And that’s resolving a problem. And knowledge is created there.
David Deutsch
Yes. And it involves a lot of conjecture. It involves a lot of false conjecture. Sure. And that is not written in the proof. It’s not in the proof. You’re guessing, what you’re really guessing in the case of learning existing knowledge, you’re guessing what was the problem situation in the author’s mind. The problem situation that he tried to write down, but wrote down that you can never write that down fully or even, you know, partly fully. You write down basically a set of clues that will help the next person maybe to get there faster than you did. Hopefully.
Eli Tyre
I mean, so far it’s working. Civilization is like, we don’t all have to completely reinvent calculus each time. It seems like we’re able to train people in calculus without them needing to do what Newton did.
David Deutsch
Yeah, well, like with many things in the educational system, learning calculus usually means learning the algorithm. Yeah, fair enough. And never understanding it. People get to get to university and then they see that the understanding that they had of calculus was wrong and the things that they thought were sort of self-evident were actually false. And then they learn this kind of better set of things for how to pass university exams. Then they become graduate students and then they’re like, holy shit, that was all false. And eventually you realize that calculus, like all mathematical ideas, like all ideas, they’re not, they’re not, they don’t come from anywhere. They are conjectured. They are calculus, for example, arises out of problems. It arises out of problems like Newton had. What on earth does it mean to have a velocity at an instant? A velocity means rate of change of something, rate of change of position. Rate of change means the difference between one position and another position. So it inherently involves two different positions. So how can the instantaneous velocity at one position have a meaning? And he struggled with this idea. Meanwhile, Leibniz was struggling with the same idea across the continent. And they weren’t trying to find an algorithm. They were guessing. Yes. And most of the things they guessed were flat out wrong in the sense that they didn’t even solve the problem that these people had. Finally, they found ideas that did solve the problem. But later generations of mathematicians were like, okay, it worked, but that doesn’t mean it was true. And in the 19th century, people worked hard to try to construct something that was true. In the 19th century, what they thought would clarify everything and get rid of these possible mistakes is by proving everything. And so they evolved better and better methods of proof until in 1900, the great German mathematician Hilbert, David Hilbert, put out a challenge for mathematicians to demonstrate a method of proof where if something passed the test of being provable by that proof, that it was true. And if it didn’t, then it was false. And Gödel proved that that was impossible. Yeah, you can’t do that. So there we are. There we are. In mathematics, you can actually prove that mathematical ideas don’t come from anywhere.
Eli Tyre
I’m not sure that I’d buy that in full generality. You can’t show that a Turing machine, any given Turing machine will halt because you could have an algorithm that asks whether or not it halts. And then if it doesn’t, keeps going and then feed that to itself. It won’t halt. You can’t show that it won’t halt. Yes, excuse me. Yes. In general.
David Deutsch
So therefore, although there’s such a thing as valid programs and invalid programs, there is no program that can tell the difference.
Eli Tyre
Okay, which is akin… And that is akin to there are true ideas and false ideas, but we can’t distinguish between them? There is no reliable process for distinguishing between them.
David Deutsch
What we can do is find errors. Which is sort of like noticing that some particular machine halts.
Eli Tyre
Yes, on a particular input. Yes. Yes. That’s a good point.
David Deutsch
Having an explanatory theory that a certain program won’t halt. For example, if the program is finding prime numbers, you can actually form a theory of prime numbers, which tells you that that program won’t halt. But that theory of prime numbers depends on the theory of integers, which Gödel proved cannot be proved consistent. So it’s a conjecture. It seems like there are simpler cases. You could posit a Turing machine that just…
Eli Tyre
Its only instruction is it takes whatever’s in the current register and moves over one, and then takes whatever’s in that register and whatever it is, it moves over one. And I can look at that. I’m like, I’m pretty sure that’s not going to halt because of… Yes. Very obvious. But the axioms that you use to prove that, and I’m not going to go into that, but the axioms that you use to prove that.
David Deutsch
Okay. A particular, pretty straightforward application of the axioms, but trying to prove the axioms is impossible. So the axiom is for any integer, there is an integer one larger than it. And the process that searches through the integers will not find a last one. Yes. Okay. And those are the axioms that set up the arithmetic of the integers. And we think it’s true. We’re sure it’s true. It is true. But what we can’t do is derive that it’s true from anywhere. Yes. All right.
Eli Tyre
So just summarizing where we’re at. We have some ideas. Yeah. So first of all, working out some computations is not the same and does not necessarily lead to getting some new ideas. Yes. At least for human beings, although… I’m just, I mean, I have some footnote there of like, would other… Would it be different for other entities? Understanding something is not the same as deriving it.
David Deutsch
Yes.
Eli Tyre
Also, incidentally, we can’t really derive anything from the root, the very bottom, because there’s no bottom that we can use. Somewhat unfortunate maybe, or at least disappointing, but you know, we humans are figuring out how to deal.
David Deutsch
It’s the only thing that makes life worthwhile, yes. All right.
Eli Tyre
Well, it sounds like you don’t think that it’s disappointing. It’s hard for me to visualize the, like, I’m not even sure exactly how, like the world where there is a bottom might be just very radically different from our world. But okay, we can’t derive anything from the bottom. I think, I do think that that’s true. For a number of reasons, which are maybe all actually isomorphisms of the same reason. And so the thing that we’re calling knowledge in this context is conjecture, where we’re starting from where we make some guess, we… Okay, so actually we make a bunch of guesses, and then we look at the guesses next to each other and we’re like, well, starting from this one has logical contradiction, maybe not necessarily logical contradiction.
David Deutsch
We’re born with guesses. Like, yes, we don’t make them all. We have some inborn ideas, and they have inborn, they have conflicts with each other. So we’re born with problems.
Eli Tyre
Can you give me some example? So I also think that we are born with guesses. But can you give me some examples of the sort of thing that you’re thinking about?
David Deutsch
One of Popper’s examples is that a baby is born with the expectation that there will be a breast there and what it will look like.
Eli Tyre
Yeah, and what it will look like. I’m not sure if that’s encoded actually, but…
David Deutsch
Well, what it will feel like or whatever. I mean, some stimuli make it go towards that and do the right thing, and some make it go away from it and do a different thing. And this expectation is never perfectly met. And the baby has also other expectations and other knowledge. Some of it inexplicit. I mean, all of it inexplicit, but some of it not directly encoded. So immediately…
Eli Tyre
I want to ask what you mean by it’s not perfectly met. The expectation is not perfectly met. Is that because there’s like a very… We couldn’t mathematically crisply describe what the expectation is. No, no. Like… So if… Let’s say that…
David Deutsch
I mean, we don’t know how any of this works. So, you know, I have to just make up stuff about how to make a baby. We couldn’t make a robot that behaves like a baby. So one thing is that some of these expectations are also desires, desires for certain sensations. So when it’s drinking milk, it is not satisfied with the state of drinking milk because that whole process involves being unsatisfied so that you drink more. Okay. And then you come to a place where drinking milk is actually unpleasant. Because you’re full. So you’re… Yeah, when you’re full. But it’s not exactly when you’re full because the physiological mechanisms aren’t perfect either. They are just a guess as the evolution made as to what would be best for babies. And that’s not perfect either. In fact, it’s a cobbled together compromise. Because the sensory apparatus isn’t good enough to detect the exact criterion of when the digestive system is, you know… So anyway, at the end, you have a conflict as well. So the baby decides whether to… How to cope with the problem of the pleasure of getting more milk with the discomfort of being full. And at some point, I think sooner rather than later, with something like a dog, they have similar ideas for newborn behavior. And they have also an inborn criterion for choosing between them. So if you get, you know, 100 different puppies and put them in exactly the same situation, they will start and stop drinking at exactly the same moment. And if it’s the same moment, you know, you will be able to see what the difference was in the environment that triggered the one response rather than the other. And that’s why there is such a thing as a book on canine newborn behavior. Okay.
Eli Tyre
And you’re claiming this is not the case for human infants.
David Deutsch
For humans, it’s already… I’m guessing it’s already not true for babies. You know, I haven’t got strong evidence either way, but I would guess…
Eli Tyre
Certainly it’s possible that there’s also an inborn criterion for this specific thing. There might be. Of balancing the pleasure of eating and the discomfort of being full.
David Deutsch
Yes. But the thing is that babies… At some point, very early on, they do things like learn the meanings of words. And there we know that there can’t be an inborn criterion because… Well, at least it’s very hard to imagine that there’s an inborn criterion because everybody learns different meanings for all the words and for the grammar. And so no two people grow up learning the same language.
Eli Tyre
What is the problem that is solved by learning language?
David Deutsch
Oh, well, a baby learns to improve its environment by understanding and speaking. So before it can understand and speak, it can’t say that it has a stomachache. It can only scream, but it would scream for many other reasons as well. And people have to guess what the reason is. Once it can speak, it can form a theory. By the way, note that it doesn’t know that it has a stomachache. It guesses that it has one. But it can form a guess and it can correct errors in that guess. And it doesn’t know in the sense that none of us really know anything. I’m saying that there isn’t a deterministic connection between various states of nerves in the digestive system and various states of preference in the brain. Again, there’s only a loose connection which is initially formed by evolution. But it’s crude. It’s very crude. There’s things like referred pain. You can have pain which is actually caused by one part of the body that appears as a pain in the other different part of the body just because the brain is not in the same body. It’s just a crude lashed up system.
Eli Tyre
I think there’s maybe two things there. I could be maybe hearing you to say at least one of two things. I was surprised to hear you say that it wasn’t deterministic. It seems like ultimately all of this has to be able to be described in terms of the interactions of particle fields which are deterministic. Yes. It’s taking many worlds for granted.
David Deutsch
Note that you can have a perfectly true theory of particles and it will tell you nothing about the behavior of babies.
Eli Tyre
It seems like if I have a detailed enough model of both babies and a theory of particles, I will get something about the behavior of babies.
David Deutsch
In order to do that, you would have to have a model of the knowledge creation process in a baby which would be a baby. It would be a person. That is sufficient. Definitely.
Eli Tyre
It may even be necessary. So that baby… Okay. Hold on. Let me say that because part of the thing that a baby is doing is creating knowledge. In order to predict what knowledge it will create, you need a process that itself creates that knowledge and therefore your computer simulation for predicting the baby is itself a baby.
David Deutsch
Yes, and can’t be predicted. And can’t be predicted. Why is that?
Eli Tyre
It seems like suppose I took all the particles in a baby. Well, we may have some problems with that. We take all the particles of a baby, we measure where they are and where they’re going. There’s where our problems are.
David Deutsch
At this point, you still don’t know, right? You still can’t predict what the baby will do. You have to put all those atoms, their positions and everything into a computer.
Eli Tyre
Yes, that runs physics forward.
David Deutsch
You still don’t know until you get to the point that you were wondering about. Why is it doing this and not that?
Eli Tyre
So you’ve got a baby there and you can’t predict what it will do until it does it in your program. Okay, I see. I see. But I can’t predict in my program what it will do before it does it. But I could run the program like if I take two literal to the particle level clones of two babies, and I run one of them forward in the same conditions and then see what happens at time step n, then I can also predict that at time step n for the other baby, the same thing will happen. Is that right? First, you’ve set up a situation where predicting some baby is retrodicting another one.
David Deutsch
Yes, that’s exactly right. That is what I’m trying to do.
Eli Tyre
So basically, you’re trying to predict what the baby will do.
David Deutsch
So basically, you cannot predict the growth of knowledge because certain types of knowledge, because the process that predicted it would be the process that you’re trying to predict. And it’s only after. So the baby, call it Fred. Fred is normally instantiated as a bunch of proteins and carbohydrates and stuff. But you could also instantiate that person in a computer program. It’s also Fred. There is no way in which that’s not Fred. They are both Fred.
Eli Tyre
Although I might say Fred one and Fred two just to keep.
David Deutsch
If Fred one and Fred two behave the same, which we are assuming, then they are the same person.
Eli Tyre
Yeah, I’m on board with they’re the same person. And you’re like, well, you can’t predict what Fred does. In general, you can’t predict what Fred does because in order to do that, you need to just watch what Fred does.
David Deutsch
Yes. And then you can retrodict it.
Eli Tyre
Right. All right.
David Deutsch
So there’s something different about knowledge creation from other physical processes. And here is where there is a weakness. If you want to find a weakness in my whole position about this. Okay. What is that thing that’s different? What’s the thing that makes creative computer programs different from non-creative ones? I don’t know. All right. That’s one of the problems of our age. Hold on. Is there a reason?
Eli Tyre
So this is also not the okay. So the reason why it is not the case. So I can look at a bit of Python code and be like and work through the logic and predict what the Python code will output without running the Python code itself. And this is running it in your brain.
David Deutsch
Yeah. Unless you form an explanation of what it’s doing. Then you might be able to predict what it’s going to do without running it. Yeah.
Eli Tyre
So I can either sort of work through the code step by step and be like, and now this variable goes up the for loop and goes down again. And I sort of keep keeping count maybe on paper. And that’s just running the algorithm in my head. Yes. I could also do something which is more like a mathematical proof and being like, well, if I’m starting with these conditions, I know that the output can’t be outside of these bounds. And that is a kind of prediction of an algorithm, which is possible for my Python program, at least in some context. But it’s not possible for Fred because there’s no way to prove what Fred is going to do. Short of simulating actual Fred.
David Deutsch
Yes.
Eli Tyre
That is my guess. I mean, I do expect particularly in the long future when we are much better at just instantiating AGIs when we want to, this thing where I can predict some instantiation of an agent by retrodicting what another instantiation of an agent did is pretty practically relevant. It means that it is in practice, I can have a deterministic model. Actually, that’s not quite right. Given some conditions or some inputs, I can have a deterministic model of how an individual develops through time. And that may become relevant as this discussion progresses. But as it stands, all right, so it is both the case that human beings are deterministic. They’re defined by the laws of physics, at least as best we can tell, but I’m not even sure exactly what it would mean if they were not.
David Deutsch
Indeed.
Eli Tyre
But also, you can’t predict what a human does without just having the human do the thing.
David Deutsch
Yes.
Eli Tyre
At least in full generality. Sometimes I will, if I’m going to poke a person with a needle, I predict that they’re going to make a sound.
David Deutsch
It depends, yes. But as I said, many of the things that humans do are not knowledge creation. So long as a human is doing something that isn’t knowledge creation, they’re much more predictable.
Eli Tyre
Right, okay. Because humans are animal mechanisms and also have some knowledge creation machinery.
David Deutsch
Yes.
Eli Tyre
Let me think if there is a counter example of predicting knowledge creation. So one thing that I might do is I’m like, well, Moore’s law has held pretty well. I’m predicting that we will somehow, I’m not sure exactly how, of course, because if I did, then I would make a lot of money, probably. I predict that somehow we will figure out how to get even more transistors onto these chips. I’m pretty confident in that prediction. I mean, I haven’t been paying that much attention. I hear that Moore’s law may be failing lately, but that’s a kind of predicting the growth of knowledge, although it’s in vague terms. I’m not saying specifically what knowledge will create. I’m saying we’re going to create knowledge which allows us to do something in this category.
David Deutsch
Yes, so there are aspects of the situation that you can guess will not be affected by the growth of knowledge. Like, for example, you know that the bits will not be stored in something smaller than atomic nucleus, so something has to give by the time we get there. Also, you know that the way that progress has been going, basically, it has been an implementation of the same idea in higher and higher resolution. So the idea has been an array of bits of memory that the memory is divided into bits. They are addressed in certain ways. There’s a microprocessor which has certain elementary operations, which we know as a matter of logic what they are and how they would fit together. So the problem of making more transistors on the same chip or smaller transistors is a problem of just implementing things smaller. So you could guess that originally when they used optical systems to photographically change something which then allowed acids to etch into the silicon or however they did it originally. Later, you could predict that, well, you could get a higher resolution if you used ultraviolet light instead of visible light. Then you can guess what it would take. Are there any fundamental reasons for it? And so on. So to the extent that it’s a purely, well, I want to say a purely engineering problem, but that’s to denigrate engineers. I mean, they have to be creative as well.
Eli Tyre
They’ve got a lot of problems.
David Deutsch
Yeah, but they, right, but the problems they are solving is how to do it. Yes. Rather than what to do. So you can guess that that’s not such a big deal.
Eli Tyre
But as opposed to, for instance, inventing the computer in the first place.
David Deutsch
Yes.
Eli Tyre
If you’re forecasting the future and you’re like, if you don’t account for, we’ll have artificial computers. That’s like a big difference.
David Deutsch
Even the scaling down thing didn’t go smoothly. There were various places where there were holdups. There’s a holdup right now, I believe, that none of the big companies are managing to get the bugs out of whatever the latest thing is called. Okay. And so, you know, Moore’s law has sort of got a bit of a glitch if you look microscopically. Predicting Moore’s law is not a case of predicting growth of knowledge.
Eli Tyre
At least in the sense that you mean it here.
David Deutsch
Yes.
Eli Tyre
I mean, like there is an important sense in which we’re predicting something that I might call the growth of knowledge. But there’s also an important distinction. So the big picture this is pointing towards, is whether we can say what we can say and whether we can say certain things about the behaviors of knowledge creating systems whose design we don’t know yet. Yes, that seems right. I mean, I think we can know some things about them. But yes, that is where we’re going. At least I hope so. Okay, so babies have some inborn ideas. And those ideas have some important ideas that are in conflict. Humans start out with some ideas in conflict.
David Deutsch
Well, that may be misleading with the thing. We have some ideas which come into conflict as they are executed. They come into conflict. I’m not sure that static ideas can be in conflict.
Eli Tyre
Yeah, okay. Fair enough. And also, I’m not sure if this is relevant, but you’re claiming that that doesn’t happen for other animals.
David Deutsch
Yes.
Eli Tyre
They do have ideas, but their ideas don’t come into conflict in their execution?
David Deutsch
Yes. If there are two of them in conflict, two fixed ideas that we would say naively are in conflict, then there’s also a fixed idea for resolving the dispute. So they aren’t really in conflict.
Eli Tyre
All right, I think this is a little bit of a tangent. But I have sometimes seen a dog which is told to stay. And also, it really, really wants to get up. And it’s very obvious from watching the dog that it really, really wants to get up. And I would definitely suggest that there’s a conflict happening there. I’m not sure how I would assess whether or not there’s an inborn criterion.
David Deutsch
Well, you can’t just from that description. Though note that when a human is very inclined to disobey a thing, that he also has a strong reason for obeying, it often results in him not looking like that at all. Because he’s biding his time, waiting for his moment.
Eli Tyre
Yeah, there’s a way that for a human, it can be much more integrated than the dog that really wants to.
David Deutsch
Yeah, but a human can choose to react in any way. He can choose to react just like the dog if he wants to, or like a cat. Okay. Whereas a dog can’t choose to react like anything except a dog.
Eli Tyre
I think that is probably true. Although, I don’t think that dogs pretend to be other animals ever. Although if they did, that would change your mind here?
David Deutsch
No, I’m sure there are animals that pretend to be other animals, but that’s because the genes say so. Those animals can’t stop pretending to be other animals if the conditions are right.
Eli Tyre
The can’t stop is sort of interesting because it seems like humans also can’t stop if the conditions are right. Because for any given human behavior, from the laws of physics perspective, it is deterministic. Given some inputs, it’s going to produce some output.
David Deutsch
We can’t choose not to do what we choose. But we still choose.
Eli Tyre
That’s not a limitation on what we can choose.
David Deutsch
For a moth pretending to be a frog, the fact that it can’t stop doing that is a limitation on what it can do.
Eli Tyre
A moth pretending to be a frog? Oh, I see. Oh, yes. Many animals are mimicking other animals.
David Deutsch
They have large eyes on their wings.
Eli Tyre
Right. Okay. Right. Okay. A moth pretending to be a frog cannot stop. That is just embedded in its structure.
David Deutsch
Yeah.
Eli Tyre
I’m a little bit suspicious of this concept of choice. I sort of expect that choice is a hand-wavy term that we use to describe the behavior of systems that are too complicated for us to predict well. Okay. Here’s the thing.
David Deutsch
The reason why it’s vague is that it involves creativity and we don’t understand creativity. However, that it exists is the conclusion of, I think, inescapable arguments like the ones that I’ve just said about predicting the baby. The growth of knowledge, for example, is unpredictable. We know that about it even though we don’t know anything about how it works. I mean, sorry, we do know something. That’s the thing. Yeah, we’re not. In fact, we know quite a lot if you think that Popper’s epistemology is in part a theory of creativity, then we do know something about it. But we don’t know enough to program it. And that’s the important question. We don’t know.
Eli Tyre
Yeah, we don’t know enough to program it yet. It is unclear to me exactly how far away we are from that.
David Deutsch
Yeah, unclear to me too. Like I said in that article, we might be just one idea away. Yeah. Or we might be 500 years away. Yes.
Eli Tyre
And yeah, sometimes people are like, we have no idea how to build an AGI. And I’m like, I think if so, for one thing, it’s no individual human can assert what all the humans know. There might be some human right now who’s putting the finishing touches on it as we speak. Absolutely.
David Deutsch
And one day that will happen.
Eli Tyre
My best guess is it won’t be an individual human. It will probably be a large group of humans that have been working really hard with a lot of money. But it could be a guy in his garage who had some clever idea and then coded it up. And then there’s a person in there.
David Deutsch
Note that the question of what is life had plagued philosophers and scientists for millennia. And then one person, or actually two people, Darwin and Wallace discovered it. And meanwhile, there were hundreds or thousands of biologists in the world going down a completely hopeless path of cataloging insects and putting them into groups and so on. And that didn’t lead to the answer. And just Darwin found the answer, basically sitting in his armchair. They…
Eli Tyre
I mean, he also went out and looked at some…
David Deutsch
It’s told as a story about going out on HMS Beagle and finding birds and so on. But he found nothing out there that he didn’t already know when he was sitting in his armchair. The key thing that made him different from all the other people that were cataloging birds was this idea that these ideas that were already beginning in his armchair. And which he then, when he went out and looked at different kinds of… What was it? Finches, finches. He knew what he was looking for. And everyone else who had cataloged thousands of different species of things, they were looking for something else and they didn’t find it. So…
Eli Tyre
Yeah, so I certainly think this sort of thing happens, at least sometimes, where there are lone people who are doing something different than what the whole field is doing.
David Deutsch
Yes.
Eli Tyre
Often make progress where the whole field is missing it. Then there’s a question of when… What sorts of fields does that happen in? Although it doesn’t seem that crucial to me right now. I agree that it is possible that the first AGI may be built by some guy in his garage.
David Deutsch
Yes.
Eli Tyre
Okay, I guess I want to talk about creativity. So creativity is… Here’s my understanding of what creativity is.
David Deutsch
Right.
Eli Tyre
So you got a bunch of ideas and you sort of execute the ideas and then they conflict with each other and then when they conflict, there’s a problem. And I guess some part of your mind identifies that there’s a problem.
David Deutsch
Yes.
Eli Tyre
Or I’m not sure exactly how that process works, but you identify that…
David Deutsch
That’s an essential thing. The mere fact that they’re conflicting, if you don’t notice it, doesn’t constitute a problem in the psychological sense. Right. Thinking is always the result of a problem. What am I getting from this?
Eli Tyre
One thing that is new that I sort of need to reformulate around is problems are central. And we start from problems. Okay, so we have ideas. The ideas have some conflict that produces a problem. Okay, so we have ideas. The ideas have some conflict that produces a problem. Yes. We identify that there’s a problem via some mechanism. And…
David Deutsch
We may be mistaken about that too. Sure. Sometimes we’re mistaken that there’s a problem.
Eli Tyre
Well, I might say that if you mistakenly identify there’s a problem, there is a problem.
David Deutsch
Oh yes, yes. It’s just… There is a problem, but it’s not between the theories you thought it was between. Yeah, right. Yeah, that happens. You know, that’s not just a theoretical point. That happens. Yeah.
Eli Tyre
So then you have a problem, and then… Magically, creativity generates a conjecture. And the conjecture is like, maybe the whole situation works like this. Does that resolve this conflict between ideas? Yes. And you… And the process of being a human is just doing that over and over and over a million times a day. And at various scales. Yes. And there’s a posit here, which is we don’t know anything or almost nothing about the process that generates the conjectures.
David Deutsch
Yes. I think most of what we know is negative. We know how it doesn’t work. We know certain things about how it can’t possibly work. And Popper is a sort of apotheosis of understanding why lots of common sense ideas about how it must work can’t be true. Okay.
Eli Tyre
So that is, I think this is maybe a tangent to this whole discussion, but that is interesting to me because my current best guess, although the way we’re using words is maybe skewed somewhat. So it may turn out that when I say this, it actually relates to something different in your ontology. But my current best guess is that conjectures are basically inductive, and that you generate conjectures via extrapolating out along various dimensions from things you’ve seen or reasoned about before. And I think that there’s some reason why that can’t be the case.
David Deutsch
Yes, extrapolation implies similarity. And similarity implies a theory of what counts as similar and what counts as different. And therefore, the content of the extrapolation is in and that theory can be false and can’t be discovered by extrapolation. So that’s why.
Eli Tyre
Okay, so I was with you until and that theory can’t be discovered by extrapolation.
David Deutsch
Yes, somebody down the street here might say that he doesn’t like Black people because Black people commit crimes. And then I could say that that’s the wrong way of thinking about it. It’s actually poor people who commit crimes. I mean, I don’t believe either of those theories, but I’m just using it as a very simple example. Now, he thought he was inducing his theory because to him, he saw Black people, Black person commits crime, Black person commits another crime, another Black person commits a crime and so on. And to him, he was thinking that he had induced this theory, whereas a different person sees the same data and he sees poor people committing crime, poor person committing crime and so on. He thinks he can induce that. Both of them have actually just been interpreting the data in the light of their pre-existing theory and have pretended to themselves. I mean, they don’t know that they’re doing this, but they have reinterpreted the thought process in their own mind as one of extrapolating a theory from the data. Actually, what they have been doing is interpreting the data in the light of a theory. And the theory came first.
Eli Tyre
Yeah, actually, what they’re doing is interpreting the data in light of a theory. Is there a reason why? So the conjecture generation process, couldn’t that be doing? It seems to me that that could be doing something like interpreting data in light of a theory. I’m not saying that it’s not theory-laden. There is theory, like background knowledge or whatever.
David Deutsch
When one does that, one isn’t creating knowledge. One may subjectively feel as though one is, as I said, because one mistakes the process as being one of getting the theory from the data. But what one is actually doing, what we do all the time, we have a theory. You’re just making a guess, though. It’s just a conjecture.
Eli Tyre
There’s still like many layers of criticism.
David Deutsch
I mean, everything is a conjecture. But in the case where, for example, the person has only that one theory about this particular issue, then although it’s a guess, it’s not a guess about this thing. It’s a guess that he formed before, and it’s just applying now because it’s his only guess. So this guess arose at some point in his past as a result of solving a problem. But then if he solved it to his satisfaction, if he didn’t conflict with anything that he knows of, then it became part of him that wasn’t controversial. It wasn’t the subject of thinking. And then when it was used, that use also isn’t thinking.
Eli Tyre
I do feel like we’re drawing some tight lines around thinking here, which I’m not sure if I would, on reflection, endorse. Creative thinking.
David Deutsch
I’m using whatever shorthand is useful. But like I said, at some point in my past, I got the value of three times seven indelibly imprinted on my brain. And now whenever I want to know what three times seven is. Yeah, OK.
Eli Tyre
But that’s not new thinking. That’s an automatic process that is of recall.
David Deutsch
When we have a theory that’s uncontroversial, we can apply it without thinking. That’s what you’ve referred a while ago to compiling ideas. When we’re compiling ideas, that’s a very good way of putting it, because not only do we make it sort of automatic, and therefore its application is not as it used to be part of a creative thought process, it’s just a mechanical process. But we may also throw away some knowledge. We may forget some knowledge. We may forget the variable names and the subroutine structure and only retain the machine code. That happens when you learn to drive. A beginner is just thinking all the time about when should I change gear and that kind of thing. Whereas an experienced driver not only does the thing, but can’t explain to you what he’s just done. Yeah.
Eli Tyre
OK, I guess I want to summarize here a little bit. When people talk about extrapolating, they are making an error or, yeah, they’re always making an error. I’m not sure that I will need to think further to see if I think that it’s always the case, but you’re claiming at least they’re always making an error, which is they are taking some categorization schema and they are using that categorization schema to interpret some data that they have. It’s not in no way pure. You always start with some categorization schema and if you’re like, well, conjectures, then you’re sort of passing the buck and you’re like, well, where did you get that categorization schema in the first place? That must have evolved from some problem that you solved where this seemed like a good enough solution to resolve that.
David Deutsch
An inductivist would give a different answer to the question where he would say, I may not remember exactly, but it must have been itself an induction process. So the philosophers in the 19th century who were thinking about induction, they would say we have in our minds a principle of induction. Where did it come from? We induced it.
Eli Tyre
I definitely don’t think that. I think that’s stupid.
David Deutsch
It’s an infinite regress and it isn’t true. Yeah, exactly. People were quite smart.
Eli Tyre
When I say that that’s stupid, it’s like when I was reading Aristotle in high school and you could tell the parts where I disagreed that the margins would be filled up with many essays about how Aristotle did not understand economics, but that’s not because I’m smarter than Aristotle. I just have the great advantage of coming several thousand years later after the invention of economics, which I much appreciate. It seems to me that there’s a question. I’m not sure how far we want to follow this because I… These are areas that I’m less clear about myself and so I’m not sure that I can make cogent arguments, but it seems to me that in general in epistemologies, you have a problem of dealing with an infinite regress. If you’re just trying to induce things all the way down and you’re like, well, how do you know that you can rely on induction? Well, because of induction, you have a bit of a problem. And also, it seems to me in what I’ve understood of Popperian epistemology, there is a similar danger of an infinite regress where you’re like, well, we generate a conjecture and then we criticize it and you’re like, well, where did the conjecture come from? And you’re like, well, there was an unconscious process. Now, correct me if you don’t believe any of this, I don’t want to put words in your mouth.
David Deutsch
Well, I don’t think that’s an infinite regress. Okay. When you say where the conjectures come from, I mean, we know where they come from. It’s just that we don’t know the implementation details and they turn out to be crucial in this case. But at the lowest level of implementation, they must be random. They must be random. I mean, not random with a particular probability distribution function, but they are not, they’re random in the sense that mutations are random. That is, it’s not that every genetic change is equally likely. It’s that what we mean by random in that case is that the changes are not caused by any particular aspect of the problem of the environment in the case of an animal. So it’s not that the giraffe stretches its neck to get the high hanging fruit or vegetable. And then the mutations happen. The mutations happen anyway. Whether it’s stretching or not, and they come first. And any improvement in the giraffe’s neck comes before the selection pressure is applied. Yeah. So that is absolutely the case in the case of natural selection. Yes.
Eli Tyre
It is plausibly the case in the human generation of ideas, although it is, I’m not sure that that’s the only way that it could work. Like the thing that I’m hearing you suggest is that when you’re trying to solve a conflict between ideas, you start with basically a random seed. And then you’re like, I’m just going to try something random. Does that solve the problem? It makes all the difference that where in the process this random, supposed random number happens.
David Deutsch
It’s not that you start with the random seed. It’s that you start with say a very high level heuristic, which is, which is then unsatisfactory in some way. And so you want to modify it. And then you’re like, well, I’m just going to try something random. Unsatisfactory in some way. And so you want to modify it. By a heuristic modifying heuristic and so on down to, you know, by the time you’re 11 stages down, then you have somewhere where you don’t have it heuristic. So that there’s some process that says, well, let’s try the first thing that comes to hand. It’s not that it’s a random number generator. It’s right. Okay.
Eli Tyre
You’re starting, you’re starting with background knowledge, but then you tweak the background knowledge to see if that works better. And then you continue a process of sort of, yes, in pruning, tweaking and pruning the high level, the higher level tweaks are dependent on the problem. Yes. Okay.
David Deutsch
It’s just when eventually after 11 level 11, you get down to a place where you’ve exhausted all the possibilities of extracting it from the problem. So you just take the first thing that comes to hand.
Eli Tyre
Oh, wait, I didn’t understand that sentence. You get down to so level 11, is there a specific level 11 or you’re just like, I’m just, no, I mean, I’m sure it’s not levels either.
David Deutsch
It’s, it’s because it’s, it’s much more opportunistic than that. Like at the highest level, you’re, you’re thinking, somebody says to you, do you know that there are actually white criminals as well? Okay. And that, and you say, no, your first thought is, no, that can’t be. And then you think, well, actually I have seen newspaper reports. Okay. It’s part of a conspiracy. They’re just saying that. All right. So at this higher level, the nature of the problem is, has a large input into the nature of the conjectures to solve it.
Eli Tyre
When you say a higher level, you’re referring to, like at the level of human thoughts, as opposed to the level of the underlying mechanisms that are generating the thoughts. Yes. At the level of explicit thoughts. Got it. Got it. Of less explicit and so on.
David Deutsch
And there, yeah. And then, yeah. And then, if the problem isn’t solved by the application specific considerations, like it, at the highest level, you’re looking, you know, that you’re looking for something that contains phrases like white people and black people and crime and so on. And then at the next level down, and you quickly scan through sort of various ones of those and see that that kind of thing isn’t going to solve your problem.
Eli Tyre
All of the solutions that are all of the, yeah, all of the conjectures that are made out of those building blocks don’t answer the question.
David Deutsch
Or all the ones that you’ve tried, all the ones that seem promising to you. Yep. Okay. Then there were various strategies, you say, well, let’s generalize the problem. Yeah. Okay. I’m not, I’m not talking about all the criminals, you know, so you can have a more general thing that tries to hedge your theory. And then if you’re rational, you’ll think, well, wait a minute, am I using a hedging process that would, that would save any theory? And if it would save any theory, then it’s not rational to assume it. Yeah. But you might not think that you might like those theories. You might settle on a theory like Marxism that explains any situation in terms of the same reason. And you might be satisfied with that for a while. You might be satisfied with that for years, or you might not. Yep. And, but what I was referring to happening at level 11 is that if the thing is still unsatisfactory when you’ve tried everything, every kind of way of changing things and every kind of way of changing the criterion and all that sort of right down to inexplicit levels, then what will happen is that the process will just try something. Maybe people are actually ducks. Maybe people are actually computers. Yeah. Ah.
Eli Tyre
That one is actually on point. Okay. All right. So my, here’s my story. Humans going about in the world, when they go about in the world, they have ideas. The, when they execute the ideas, they are in conflict with each other. That causes a problem. Some mechanism in a human mind recognizes there’s a problem and initiates a process. Maybe the process is actually going on in the background all the time, but a process either starts or continues of generating conjecture that attempts to resolve the conflict between ideas. That conjecture process. So first of all, we don’t really know how it works, but here’s a conjecture about how the conjecture process works. Yes. We, we have a bunch of background. So, yeah, so now we’re in the box of this is the conjecture generation procedure. We have a bunch of background knowledge and we will start by just applying our background knowledge and seeing if that solves our conflict. And, you know, we, we tried that a bunch. And then if that doesn’t work, that doesn’t resolve the conflict. Sometimes it does. Sometimes you’re like, ah, you just forgot about regression to the mean. There it is. Yes. But if that doesn’t work, then you might sit there and sort of modify your background ideas until you get a version that you’re, then you’re trying the modified versions. You’re like, well, maybe it’s not exactly. Yeah, we were using example of racism where we’re like, well, okay, maybe it’s. I have some idea that only black people commit crimes. And then I am like, okay, well, maybe it’s mostly black people that commit crimes. And I’m like, does that solve the, yeah, I’m sort of, I’m sort of tweaking it a little bit. And then I’m continuing. So level one is I just try my background ideas. Level two is I am where again, levels is sort of like the order in which we’re doing things. Maybe. You’re changing your initial theory.
David Deutsch
Yeah. You’re changing it intentionally to meet, you know, to meet this problem.
Eli Tyre
It’s not a random change yet.
David Deutsch
Yeah. You’re thinking this, this kind of thing might solve the problem. Right. So I do want to flag that in terms of like in the project of designing how an AGI works saying we are intentionally doing it to solve this conflict sort of leaves open like a I can’t write code that’s like now come up with a solution that seems like it matches this conflict between ideas.
Eli Tyre
Let me just say at this point that conjecture is only step two.
David Deutsch
So recognizing a problem is step one, conjecture is step two. Then there’s criticism. We haven’t mentioned criticism yet, but at each point when you notice that tweaking the problem to saying that maybe it’s just mostly black people, noticing that that doesn’t work is a critical process. And the critical process in general is in general also creative. So you’re creating usually you’re not using stock criticisms. You’re usually creating a criticism. Sometimes a stock criticism will do like you said, you know, if you have a list in your mind of your biases that you have, then you can say, well, would undoing this bias do it? Well, that bias. So but also in any kind of non-trivial thought process, the critical phase is also conjectural, creative.
Eli Tyre
And when you say it’s conjectural, you know, that means I have some conjecture, which I’m putting forward that maybe resolves the conflict between ideas. And then there’s another process which is going to generate a conjecture for why our first conjecture might not work. Yes. And then I guess we also need to critique our critique, right?
David Deutsch
We’re going to generate like so you can form the theory that your critique has a flaw or you cannot form that theory that the number of possibilities increases exponentially as you go along. And what you actually choose is only a small proportion of those. And which ones you choose are determined by heuristics of how you choose those things, which are themselves theories, conjectures, which can be changed.
Eli Tyre
I didn’t capture that last sentence, which are themselves conjectures, which can be changed.
David Deutsch
Yes. So your critiques are themselves conjectures, which can be changed. Yes.
Eli Tyre
So all of this gives sort of the impression of a very busy mind. There’s just like a bunch of pieces that are all both like there’s some conflict and we’re going to generate a conjecture and we’re generating a counter conjecture that will maybe critique the conjecture.
David Deutsch
And we ain’t scratched the surface yet. Yeah, right. But just I was just pointing out that the process I’ve described grows exponentially because at each stage you’ve got the choice of whether to reject the conjecture on the basis of the criticism or reject the criticism on the basis of the conjecture or form a new one of either of those two. So there’s like four possibilities at each level. So by the time you’ve got to level 11, it’s four to the power of 11. And there’s not room for that in the whole brain. So you need a pruning system. Yes. And that pruning system is itself conjectural knowledge, which is being changed, which is subject to change.
Eli Tyre
Okay. So on checking that you said there were four cases, you can accept the conjecture, you can accept the criticism of the conjecture, you can modify the conjecture, or you can modify the criticism of the conjecture.
David Deutsch
Well, you can start trying to modify them. Yes, but by the same recursive. Right.
Eli Tyre
Yeah, we’re actually we’re calling our do epistemology function. We can sort of double click and be like, all right, now we’re going to do the.
David Deutsch
Yes. It’s like chess for a human to play chess, you’ve got to have some drastic pruning mechanism. The way humans learn chess is creative. So this thing is not just an analogy. That is exactly how we learn to play chess. We learn to prune our pruning algorithms and so on.
Eli Tyre
Okay, where I’m like, maybe this is a good move. So it seems like when I’m learning chess, I’m largely I had a conversation with Lulie about this on Twitter. In fact, I’m largely learning from feedback where I conjecture maybe this is a good move. And then I get trounced by the person I’m playing against. And then I learn turns out that was not a good move. And in that sense, the at least in that situation, the criticism part is sort of coming from, I guess, okay, it has to turn into an idea in order to operate in my mind. But the idea is, well, I lost that game by a lot. So something, something didn’t work.
David Deutsch
Yeah. But then you form a conjecture and it’s not just this was a wrong move. It’s a conjecture about more importantly, I think more often than that. It’s a question of what was wrong with my way of choosing that move? What should I have noticed? Okay. Then, you know, at deeper and deeper levels, you build up what chess players call an intuition. Yes. Which doesn’t take the form of if this happens, then this happens, then this happens, then this happens. It takes the form of this is a position.
Eli Tyre
It feels like a good attack.
David Deutsch
Yeah. Without any specific thing. And that is how we learn knowledge about chess. That’s how grandmasters could beat computer programs that analyzed orders of magnitude more cases than the grandmaster did. Yeah.
Eli Tyre
Because there’s compressed theories about what makes a good chess move.
David Deutsch
Yes, there’s genuine knowledge there. All right.
Eli Tyre
And you would say that the chess program is not genuine. The chess program, the chess engine, all it does is draw out implications. No knowledge creation is happening.
David Deutsch
Yes.
Eli Tyre
Although, I mean, there’s knowledge creation in the discovery of game trees. Like what you need to do to build the chess engine is definitely creation.
David Deutsch
Definitely. Yes.
Eli Tyre
Although, I don’t know how familiar you are with. So it seems to me that I would say that AlphaGo does do knowledge creation. Yes, it may.
David Deutsch
If so, it’s evolutionary knowledge creation. It doesn’t know what it’s doing. Yes. Yes. Like evolution.
Eli Tyre
But it is doing the same thing of playing out a game tree and then learning from that which sorts of moves seem like good moves overall.
David Deutsch
Yeah, well, biology does that too. At some point, it changed from using RNA to using DNA because DNA is more robust. But it didn’t know what it was doing. There was no instantiation of that explanation that I just gave in the process. It was just dumb evolution. Creating knowledge very slowly. And AlphaZero creates knowledge very, very, very slowly. It’s just that we don’t know.
Eli Tyre
In terms of the number of games it plays.
David Deutsch
Yes.
Eli Tyre
Yeah, so I definitely, I did… I once made a… Yeah, there’s something you might call sample efficiency, which is how much Go skill do you get per game of Go that you play. And by this measure, AlphaZero is very, very bad compared to a human.
David Deutsch
Yes.
Eli Tyre
But AlphaZero makes up for it by playing way more games than a human has a chance to play in their lifetime.
David Deutsch
In terms of specifically playing chess, it does. But the knowledge that a pro chess player has enables all sorts of other things to happen as well, such as explaining what he’s doing. So a chess player can write a book saying how to play attacking chess. And AlphaZero, even though it attacks better than he does, can’t write anything like that.
Eli Tyre
That seems correct to me.
David Deutsch
Whenever you have something that can learn from apparently much fewer cases, it’s because the programmer has put in the knowledge that will restrict you to those cases. Now, a little bit of that happens whenever you get an ordinary AI learning things, in that when a human learns those things, because they’re learning explanatory knowledge, the knowledge has reach. And they have also learned something about, for example, when they learn chess, they’ve also learned something about games in general, about making certain kinds of decisions in general and not others. And that is because they are not learning, when they learn chess, they’re not just learning how to win, how to win at chess. AlphaZero is learning how to win at chess, but it’s not learning, for example, how to make a beautiful game. Now, because of the way that chess players learn chess, they’re learning these heuristics, which are connected with heuristics about beauty and other things. And so you often hear chess players saying that, okay, I won, but it was boring. And AlphaZero never says that. And in fact, well, I think it might be possible and should be done, should be tried, to make AlphaZero not only win games, but report whether they were boring or not. And this is a much more difficult problem, but I think it’s possibly within the scope of dumb AI. By the way, I’m not convinced that AlphaZero is like biological evolution. That’s kind of the most I can imagine it being, but I would guess that it’s dumber than biological evolution.
Eli Tyre
Seems to me, I have heard it said that you have said, although I don’t want to put words in your mouth, and maybe you’ll be like, I would never say something like that, that you think that a superintelligence is impossible because there’s universality of explanations or explanatory universality. And so there’s sort of like two categories of things. There are things that can generate explanatory knowledge at all, and there are things that cannot generate explanatory knowledge at all. Yeah, so first I want to see if that’s about right, and then I have a response to that.
David Deutsch
Right, these things in question being programs.
Eli Tyre
Yeah, although also we can specify pretty much, we can think of anything as a program, right? They’re all, but yes.
David Deutsch
Well, it’s just that people say, is there something which is to humans as humans are to ants? Well, before answering that, I have to rephrase it in the form, is there something which is to the program running in human brains as the program running in human brains is to the program running in ant brains?
Eli Tyre
Okay, yeah, I am solid with that.
David Deutsch
Yeah, you see that it’s just conflating two different kinds of difference. One thing about an ant brain is that it only has a few K of memory and processing space, and humans have many gigabytes at least. There could be something that has many exabytes, which is vastly more, let’s say, and there certainly could be a thing like that. Yep. In fact, there is a thing like that, a human plus the internet is that.
Eli Tyre
Yeah, although also there’s an important difference if it’s in your head versus accessible to you on the internet. If I could modify myself to have exabytes of memory, I think I would be like, that would make me quite different as a person than if you just gave me access to the internet, which is also a really big improvement. I don’t want to sound ungrateful for the internet.
David Deutsch
I mean, in a way, you’d be a better person, like when they invented writing.
Eli Tyre
Different person, not necessarily better. Well, a different person.
David Deutsch
So I agree with Nick Bostrom that technological improvement will enable ways of being that we don’t know about now, that we aren’t conceiving now, and which are in fact better than now. Okay, great. Those people would be better than us in a certain sense. But that is not to say that the ideas they have will be some kind of different thing from our ideas. We could have those ideas just by plugging in some add-ons to our brains.
Eli Tyre
Yeah, so I think I broadly agree with that. I don’t think that superintelligences will have some fundamentally different sort of idea than humans and especially enhanced humans would have. There are some important details about how that enhancement can work that I would want to work through. Yeah. It seems to me though that we were just talking about a whole zoo of conjecture processes and critique processes and fractal conjecture critique processes and pruning processes that are happening in order to generate even a single conjecture for given conflict between ideas in a human mind. And as we say, we don’t know how all of that works in detail. It seems to me like there is probably a class of algorithms that basically do those functions and that some of those algorithms are better or worse than other algorithms. And therefore, you could write a mind which is much better at doing epistemology, at getting better explanations faster and a mind that is much worse at that based on the algorithms that you’re using to do all of these different functions. We already have that.
David Deutsch
So the key concept is subroutines. We call subroutines and we know that many of the subroutines that we consciously call are executed unconsciously. It wouldn’t make any difference to us if they were executed unconsciously in an add-on. It would feel exactly the same.
Eli Tyre
I sort of want to hold off on talking about add-ons for now, although I do agree that in principle that that’s a thing that we could, that humans could do that and it is desirable that we do that. But I want to start by just clarifying that it is possible to have a thing which is effectively much smarter than modern humans because the algorithms that it’s running to do epistemology are better in some ways than the algorithms that humans run to use epistemology.
David Deutsch
Well, I still think you’re kind of having the wrong picture of what it takes to be better. We have, you know, I just said our unconscious mind is an example of an add-on. Pencil and paper is another example of an add-on. We, there’s no, our, we’re not a different person by getting a better add-on until we decide to improve as a person by learning to choose those unconscious processes or processes in the add-on rather than others. Okay. Once there are AIs that are, have developed the knowledge to use vast amounts of memory for thinking, which I think will not happen that fast, by the way, I think that’s not the, we are not memory limited or speed limited. The bottleneck in human thinking is overwhelmingly, you know, software, but okay, that’s a different issue perhaps. But by the time AIs have that, humans will also have it. It’s just additional hardware that humans could also use.
Eli Tyre
Okay. So, I want to flag there’s a by the time claim, like now we’re not just talking about in principle what things are possible, but we’re talking about technology, and we’re talking about technological trajectories and which ones will reach fruition at different points in time, which is not fully a crux for me, but is a big chunk of a crux for me. If I thought, if I thought that human augmentation, strong forms of human augmentation was going to arrive before AGI, I would, I don’t think this fully solves the problems that I’m concerned about, but it does seem like plausibly we’re in a much better situation with regards to these problems. Thanks for meeting with me for two hours.
David Deutsch
Well, thanks, it’s been really fun.
Markdown