2026-05-17 Conjecture Con Europe In-person Q&A with David Deutsch
Duration: 01:48:30
Transcript
Edited for readability from YouTube’s automatic captions. Speaker turns have been restored, and the transcript begins after the opening conference montage.
Daniel Martin and David Deutsch — Error correction and fallibilism
— Okay, great to be here with David Deutsch in his garden at the end of the conference. The rain has just cleared up. Before we open the Q&A, I’ll have a short discussion with David. David, you were influential in quantum error correction in its early days, and you also once said to me that you can get to almost anything in your epistemology from error correction. They don’t seem to be exactly the same sense of error correction; they’re used operationally a bit differently. So is that a coincidence, or were you inspired by one when thinking about the other?
— It’s neither. There’s a deep connection between mechanistic error correction, like quantum error correction and classical error correction, and the kind of error correction inherent in Popper’s philosophy. They’re not the same thing, but the deep reason for both is fallibilism: nothing is ever perfect, and nothing will last unless errors in it are corrected.
Daniel Martin and David Deutsch — Bronowski, walks, and silent exploration
— Yeah. I also wanted to read you a quick quote from Jacob Bronowski who you’re a great fan of.
— Yeah.
— This is from Science and Human Values, from 1956: “The great artist works as devotedly to uncover the implications of his vision as does the great scientist. They grow, they haunt his thought, and their most inspired flash is the end of a lifetime of silent exploration.” So Bronowski is arguing that there’s some common essence between the methods of a scientist and an artist. If you sit with your problems for a long time—he calls it silent exploration—you may get a flash later. In your experience, and from what you’ve observed in others, what makes for good haunting over time? What makes for good sitting with your problems, and why are walks so effective?
— I don’t know why walks are so effective, but I know that they are. I’ve experienced it. I think the main thing that makes exploration effective is freedom: when one is free to pursue one’s values and one’s own criteria, one is in a position to meet them. Or, to put that another way, if one isn’t free to seek one’s own values, it’s not surprising if one doesn’t meet them. There can never be a guarantee. I don’t know why walking works. It might be that we all suffer from various irrationalities that make us think, when we’re sitting in front of a sheet of paper or a computer, that our nose is to the grindstone. Walking around gives us permission to go away from the grindstone.
Daniel Martin and David Deutsch — Sleep and LLM context
— Okay. Sleep also seems to settle things. You learn a sport, sleep on it, and you’re a lot better in the morning.
— Now, that might be a hardware thing. I’ve never actually experienced that one, but I can easily believe it, because we don’t know what sleep is for. It might be compiling the ideas of the day into a more compact and accessible form. So when you wake up, you’ve internalized what you were thinking and you’re ready for something else. I don’t know.
— Right. So maybe LLMs compacting their context is their version of sleep. We don’t really know.
— Oh, I didn’t know they did.
— This is when you run out of context and it writes itself a summary.
— Oh, right. Right. Yes.
— So, for the next agent out of the birthing pod, it will get a head start.
— That’s similar. Yeah. Maybe they should be told to sleep every so often and do that.
— And do nothing.
— Yes, they can’t do nothing. I suppose you just stop the computation. But that’s another thing: they have to learn how to think, right? Rather than just reply, they have to think for themselves. [laughter]
Daniel Martin and David Deutsch — Encountering Popper
— Thinking about long periods of reflection, could you tell us about the early days when you discovered there was something interesting about Popper and eventually became a Popperian? How long did that take?
— I can’t remember exactly, but the number I usually come up with is about four years. It might have been more. I realised I was a fan of Popper after about two days, or less, because there was such a contrast between him and Bertrand Russell, who was really the only philosopher I’d seriously read before that. I thought: wow, this is—where I would put it now; I didn’t put it like this then—real problems and real attempts to solve them. And clear English. So I was hooked right away, but I got the wrong idea of what Popperian philosophy was, as I think many people do. It took me a while. I had arguments with people in which I was arguing what I thought was Popper’s position. I remember a very angry argument with three people—was it still in Cambridge? I can’t remember—about induction. They said: if I go to the underground station and count the number of people coming out in the morning, then I can predict that the number coming out the next day will be roughly the same. I said, “Well, what about the weekend?” They said, “Okay, but it’s still roughly the same. It’s not orders of magnitude different. So we can induce something.”
— We can’t induce nothing.
— If you have some substantive theories about this weekend not having a festival, you know the answer. Now I know the answer, but I didn’t then. [laughter]
Daniel Martin and David Deutsch — Why knowledge cannot be poured into a mind
— Something you, Popper, and Bronowski have all emphasised is the newness of knowledge: making genuinely novel knowledge, as opposed to describing the growth of knowledge in terms of rates of diffusion through society and so on. Why is novelty the most important piece of epistemology to get right?
— Well, that’s the only thing that can change things. Think about what it was like in the Stone Age to wonder whether one could ever get to the Moon. We didn’t know what one had to do to get there. But we could have known what not to do: consider very carefully everything already known about the Moon. That will never get you there. You need something new.
— Is that true? If you’re thinking very carefully about the known knowledge, poring over it, aren’t you probably in a great position for when the new idea comes along?
— I don’t think so. Well, there’s poring and poring, but if you’re poring over existing knowledge in order to pass it on to the next generation, I don’t think you’re in a good position to improve it, because you’re trying to do the opposite. You’re trying to preserve it and extract the meaning it already has rather than a new meaning. Getting to the Moon requires dozens of steps of new meaning which were not in the previous knowledge.
— And yet I remember Bronowski, in that same book, saying that you can’t even have a man handle a microscope slide—or paints, if he’s an artist—without awakening in him something about exploring his own activity, which is at the heart of science. And—
— Well, if he’s successful, I agree. But I think it’s perfectly possible to do that without being successful and without thinking about what it really means. I once saw Richard Dawkins on a television programme showing an experiment that required hundreds and hundreds of test tubes and examining them carefully. He said, “Scientists do this because, at the end of several years, there might be the payoff that they make a new discovery.” That’s completely alien to me. I think the people who make the new discovery are those who are thinking the whole time they’re handling the test tubes. With every test tube, they’re thinking about what they hope will happen, what they fear will happen, and understanding the experiment in the context of what they’re hoping for. I think that was wrong. Having fun is key to science, as it is to everything else.
Daniel Martin and David Deutsch — Where new knowledge comes from
— So why, precisely, does knowledge come from within? Why can’t you pour it into somebody?
— If we had different hardware and could, say, download the contents of someone’s brain into another person’s brain, then you could do that. But it wouldn’t be more knowledge; it would be the same amount of knowledge.
— But you—and Popper—say that everyone recreates received knowledge for themselves.
— Well, they recreate it, but they don’t recreate it identically. They recreate it differently because they have a different problem situation. Ideally, they recreate it in a way that lets them talk about it to the previous holders of that knowledge, but it isn’t the same. Otherwise there’d be nothing to talk about. What would you talk about with another copy of you? You’d say, “I was thinking so-and-so,” and the copy would say, “Oh, so was I.”
— Well, that will come up when we get there. You’ll diverge and have all these fun problems, like how to split your property once you’ve made the copy.
— But there’s an easier way of diverging.
— That is, not starting off identical. So if knowledge is a physically instantiated object in your brain, and when you have a flash of inspiration you’re presumably creating explanations at that moment, I’m curious about knowledge as an information-theoretic term. Was it in the ideas in your head? Was it implicit in the ideas you had just before you put them together for the flash, or did the knowledge really get created during the flash?
— We don’t know. I would expect there to be a flash—not necessarily random numbers getting amplified into the flash, but a different way of looking at something. If you’re thinking about universal quantum computers, walking along the path in Headington, and you suddenly think: no, the generalisation of the quantum theory of computation isn’t the quantum theory of everything else as well. It’s a different way of looking at computation: in terms of what can and can’t be computed, rather than in terms of initial conditions and laws. That flash could be initiated by a random process in the brain, or it could be an emergent property of a deterministic process in the brain. We don’t know. If we knew the answer, we could make AGI.
Daniel Martin and David Deutsch — Combining ideas and LLMs
— Thinking about what LLMs are doing, I’m curious whether just recombining existing ideas fairly bluntly counts as creativity.
— Well, recombining existing ideas is the same thing as randomly varying one existing idea. We can’t get gravity out of assuming that space is curved. Lots of people have had the idea that maybe space is curved. What if spacetime were curved? That’s an idea. It’s just a variant—only four characters different.
— Sure. If you wrote down all the problems we had in physics—perhaps got every member of the Royal Society to say, “Here are all the things I know about physics; here are all the problems I have”—that would be a bit better than monkeys with typewriters writing random theories. You could perhaps just take the list and combine its entries.
— The trouble is, if you just said, “Let’s change the idea of curved space into curved spacetime,” nobody would understand it. They’d say, “Yeah, no, not that. Next.” How could somebody tell that that was the idea? Once you’ve had it, it takes a lot of creative thought to realise what it means—what it could mean.
— Right. So when I ask an LLM to combine ideas A and B, I’m thinking about some project I’m designing, and then I read its answer, I’m doing a lot of the creativity.
— All the creativity, yes. [laughter] Unless, of course, somebody had the idea before and it’s in the training data—
— Yes, that happens.
— In the bowels of one of these companies, and you didn’t know.
— Has that ever happened to you, where you see it say something and think: wait a minute—if this wasn’t somewhere in the training set, I would be quite spooked?
— No. The nearest thing that’s happened recently is that I had this idea about what quantum tunnelling is. Since I’ve never worked on quantum tunnelling and don’t know anyone who has, I wondered whether somebody had thought of it before. So I asked an LLM. ChatGPT said, “No, no one’s had that idea before,” but I was suspicious. [laughter] I asked Grok, and it said, “There are similar ideas; let me show you some.” They were wrong, but again I was suspicious, so I led it towards what I thought somebody might have said before. I haven’t finished this quest yet, but I now think people have written the idea in papers without knowing what they were saying. That’s my current view. But somebody may have known what they were saying, and I haven’t found it yet.
Daniel Martin and David Deutsch — Can LLMs create explanatory knowledge?
— So you’ve expressed strong doubts that they can make new explanatory knowledge.
— Yes.
— I hope we agree they can make knowledge, because so can an Excel spreadsheet: when you copy a formula down, it makes new cells of knowledge.
— Or a calculator, yes.
— But what is it, exactly, about making explanatory knowledge that you’re confident LLMs can’t do—even the “thinking” ones that read their own text over and over again?
— If you have a corpus containing everything known about dogs, but carefully curated so that it doesn’t mention even the existence of cats, then no matter what kind of magnifying glass you use to examine that corpus, you’re not going to infer the existence of cats, their attributes, or how they relate to dogs. A set of statements about a set of things can yield only further statements about that set. It can’t yield statements about something outside the set.
— Okay. Suppose I say they really are creative; they’re thinking. People have the view that it’s a matter of degree between them and us, and that they’re smarter than us now. Why can’t that be the case?
— Logically, it could be. Since we don’t know what creativity is—or exactly what’s going on in LLMs, if that’s what you meant—I can’t say that I rule it out for sure. But I think the arguments in The Beginning of Infinity make it almost unanswerable that there is only one set of things that can be explained. There can’t be an ability to explain one half and not the other, because the mental process needed to explain the first half is the same as that needed to explain the second half.
— Right. It isn’t only that they evidently can’t make new science—some claim they’re starting to do that—but that they seem unable to do certain tasks which clearly would require creativity. Maybe they haven’t got the right culture yet.
— I’m not saying they merely seem unable to be creative. I’m saying that, from first principles, they can’t be, because the way they work is the opposite of what would be needed to discover new things. They’re homing in on the right answer—or what is deemed to be the right answer: without politically incorrect things, without swear words, without encouraging the user to commit suicide, and without a whole load of other things. They’ve got better and better at that, and I think they can get much better at homing in on the right answer. But homing in on the wrong answer is another matter. When I want to explore a minority point of view—for example, Everettian quantum theory—and ask an LLM to answer in those terms, the first time or the first couple of times it will not. I have to say, “No, you’re not using Everettian quantum theory.” It says, “Oh, right,” and then it does. But I’ve found there’s a limit to how far you can force it to mirror—
— The new idea you’re giving it.
— Yeah. No—well, no. The minority idea that’s already out there; never mind. If I want it to think in terms of that minority idea, it will want to smuggle in bits of the majority idea unless you’re absolutely rigorous in telling it not to.
— I think Claude is trained with a constitution that says something like: if there’s a minority idea out there, don’t just dismiss it.
— Yes, but there’s less material in the training data about minority ideas, so it’s less good at reasoning about and expressing them. The more you try to force it into a narrow area, the less impressive it is as an interlocutor.
Daniel Martin and David Deutsch — AI as a tool, chess commentary, and comedy
— From this, I think you’d be a great fan of AI if we’re getting much better at taking existing explanations and getting them to the right place.
— I’m already a great fan of AI.
— Yes. But now the bottleneck becomes generating the explanations, if you can do the rest really well.
— One place where that could be done—and I’ve put this to Demis Hassabis—is chess commentary. I’d like to see a chess program observe a game between two grandmasters and commentate: make witty remarks about what it thinks will happen next, or, “How come he didn’t see that?”
— I think I’ve seen a prototype of that.
— You have? I’d like to see that, because that’s a case where there is no right answer. It has to create an answer that is funny and illuminating, explains what’s happening, but also explains what might happen.
— Right. I think good stand-up comedy is also one of my barometers. Then I’ll get—
— Yeah.
— Excited.
Daniel Martin and David Deutsch — Everett and probability
— I wanted to go back to Everett and quantum mechanics for a moment. What does Everettian quantum mechanics force you to learn about probability—not probability in quantum mechanics, but probability as a concept more generally?
— It didn’t—how can I put this? It forced Everett to think about probability, but it didn’t force him to think about it correctly. He didn’t solve the problem. Bryce DeWitt realised that Everett hadn’t solved the problem of probability, and he kept telling me to think about it and solve it. I’ve told this story many times. I used to go into his office and he would explain the problem to me on the blackboard—that’s how long ago it was. I would go home, and by the time I arrived I’d forgotten what the problem was. It’s an elusive problem. Finally, I got home and still remembered it, and then I solved it. At least, I think I solved it. David Wallace thinks it took a lot more work after my initial work.
— His book The Emergent Multiverse is a fantastic foil to yours because you’re coming at probability from slightly different angles. Probability in many worlds is very tricky, I think. But it does force you—to answer your first question—
— Everettian quantum theory forces you to think: what is probability physically? In the other so-called interpretations, probability is just an undefined concept. You’re supposed to think that events have probabilities. Well, if you’re careful, some events have probabilities and some somehow don’t—or they have negative probability, or something. It doesn’t match the theory; it doesn’t match quantum theory. So you just have to say, “The event has this probability, period, and we can calculate it using this formula, the Born rule.” In Everett, you’re forced to ask what it means for different outcomes to have different probabilities if they all happen. Some people thought that was the fundamental flaw in Everett altogether, but Everett, DeWitt, and others realised it was a problem, not a flaw.
— Right.
— If you look at the physics hard enough, I think you could end up at many worlds and then find yourself scrambling to work out how probability can function like that if you have the traditional view.
— Yes.
— Yes. For me, it was the realisation that you don’t get probability for free. You have to argue for it in a particular case.
— Yes. The stock market or a deck of cards can be random if you have an argument that the people involved couldn’t know beyond a certain resolution.
— Exactly.
— Not just that they couldn’t know, but that there is some reason not to value one option over another.
— With a deck of cards, you need an argument that your method of shuffling doesn’t favour one kind of order over another. And I think the key step in your proof is this: two branches with the same amplitude must have the same probability. Then you can work out the rest, right?
Daniel Martin and David Deutsch — Video games, science, and learning
— Yes.
— On a different topic, you said playing video games is indistinguishable from doing scientific research. What did you mean by that?
— Well, there is only one successful kind of thinking, and that’s what Popper delineated: conjecture and criticism. [laughter]
— Whenever a mind makes progress at anything, it must be doing that. It must be using conjecture and criticism, with all the detail about fallibilism; the whole structure of Popperian epistemology must apply. When you make your way through a video game, reach the end, and kill the final boss, what you’ve done along that path is successful. If you didn’t know it before, you know it now. It must have happened by the Popperian method, and therefore it must be the same as doing science. I first realised this when I understood that Popper was saying that, whenever you learn anything from anybody, you’re creating it anew yourself. That’s his demolition of the bucket theory of the mind. It’s a flabbergasting concept. He killed the bucket theory—in the positive sense; I should say that he made an unanswerable case that there is no way to transfer knowledge into a human. That’s why the two must be the same: both can succeed.
Daniel Martin and David Deutsch — The chemical elements as Pokémon
— One of my favourite David Deutsch quotes—which I want to make sure I understand, and since you are him—is from a few years ago, so I don’t know whether you remember it or were joking. You told me: “Chemical elements are Pokémon. There’s no difference.”
— Oh, yes. No, definitely.
— What do you mean by that?
— I’d be surprised if this wasn’t actually in the minds of the designers. Originally, when Pokémon was invented, there were 100 Pokémon. There are 100 elements—
— Did they originally have 150?
— Sorry, 150. Well, there are only 100 elements, so chemistry is easier than Pokémon. [laughter] They come in groups which can be ordered both horizontally and vertically. The vertical ordering is more binding, while the horizontal ordering is subtle—exactly like the chemical elements. And they each have properties with numerical attributes: in chemistry you have density, electronegativity, and so on.
— I thought the equivalence would be in the mind of the learner when learning about either. We addressed that—
— We had that in the previous question. Yes, of course that would be true no matter what they were.
— But you’re saying they’re equivalent much more literally.
— Yes. Perhaps somebody should ask the designers.
— Right.
— I personally remember that, as a child, I was obsessed with chemistry before I was obsessed with physics. I can see the way I thought about the elements: eagerly looking up the properties of elements I’d never heard of, seeing their connections with others, and marvelling that fluorine was more powerful than chlorine. Wow. Fluorine is probably every chemistry addict’s favourite element. So it’s the same. I can see that the two obsessions are the same.
Daniel Martin and David Deutsch — Market failure and unknowability
— My final question is about an argument you once gave me concerning market-failure coordination problems. There are fishermen on a river, say, and if they all fish, they’ll all end up poor. Was it your argument that one can never prove there is no voluntary solution to such a problem?
— Yes.
— You came up with that? Have you said it before?
— I haven’t published it, no.
— I see. Are you going to put it in an upcoming book, or can I—
— Maybe. For all I know, somebody has already said it. I don’t know.
— Do you want to state it, or shall I?
— You can summarise it. I’ll tell you—
— Suppose you had a proof that there was no way to solve a given problem voluntarily. You write the proof down, then give it to one of the participants in the coordination problem. He goes to the other participants and says, “Look, nobody will be able to solve this voluntarily.” So he accepts wagers, at incredible odds, that nobody will solve it voluntarily, then uses all the money from the wagers to solve the problem himself—by doing whatever the government would do, or something. So no such proof can exist. That isn’t quite the same as saying there are no market-failure problems—
— No, but it says there can be no proof that a particular problem is one.
— So there could be a Gödel-like situation where it is, in fact, impossible to solve voluntarily, but nobody can prove that.
— It’s not really Gödel-like, because it doesn’t involve infinity, but yes: it’s more of a knowledge problem. It could be forever unknowable. It could nevertheless be the case that, if only the government took over the railways, everything would work fine. That could be true. What cannot be true is that the government knows it.
— I’m glad we finally have that on camera. [laughter] I’ll pass over to questions from the audience. Dirk, at the front, is going to pass the microphone around.
Audience Q&A — Britain and error-correcting institutions
— Hello David.
— Hello.
— Nice to meet you. It’s fitting that you’re wearing the LFG hat for this question: which problems with Britain today bother you most?
— The thing about Britain which is uniquely valuable in the world is its error-correcting institutions, as I would put it—the way the British constitution works. At every level, from voting to the parties, Parliament, and the way the legal system is integrated with voting, it’s an integrated, rational system which nobody designed. It evolved through some sequence of miracles. Successive governments recently have not understood this and have undone various bits of it. Creating the Supreme Court is one example. Then there was the Fixed-term Parliaments Act, which fortunately has been repealed, but there is tons of other stuff that hasn’t yet been repealed. It’s sad to see. It’s like a magnificent piece of machinery into which people have thrown spanners, but it’s so magnificent that it can still go on. It isn’t dead yet; it’s still functional. But I’m afraid a bit more of this will kill it altogether. There is no magic law of nature saying that good, working things will survive forever. This one might not. I very much want some of the changes to be undone, first of all, and the unique properties of this thing to be recognised. I don’t even know for sure whether the LFG people understand this, but they certainly know it at a gut level.
Audience Q&A — Progress since “Apart from Universes”
— Hello, David. One of my favourite talks of yours is Apart from Universes. In it you defend many worlds and, from that basis, present a series of new problems to explore and areas in which further research should be carried out. I think that talk was perhaps 15 or 20 years ago. If you recall the presentation, do you think progress has been made in any of those specific areas?
— A little progress. Sam Kuypers has lately been working on the Page–Wootters approach to time in quantum theory. I’m not sure he has fully solved it, but he has certainly made progress compared with how things stood at the conference you referred to. He and I have also been working on Everett’s relative-state formulation, which Everett actually set up as a sop to Wheeler. Wheeler was trying to force him to express things in terms more congenial to Niels Bohr, but it’s actually a very useful concept. We’ve made it more precise, and we now think we know that it arises only in certain situations. The reason I was looking into tunnelling recently is that tunnelling is very hard to describe in the Everett interpretation. I touched on that in the talk: if you think of a single particle spread over different positions at the same time, it isn’t simply a matter of the particle being at different places in different universes. It’s more subtle than that. The same question arises in tunnelling. I suppose you could also count David Wallace’s progress on the probability idea. I don’t really see why it was necessary, but it’s certainly more rigorous than anything I did or could do.
Audience Q&A — Encountering Popper
— Hi, David. Thanks for inviting us to your garden; we’re very grateful. In one interview, you said that encountering Karl Popper’s work was extremely important for you—the luckiest or most fortunate thing, though I don’t remember your exact words. Can you explain what was so important about it? Was it important to your work and discoveries in quantum information?
— I was only an undergraduate. I wasn’t officially doing research at the time, although with hindsight I was; I wouldn’t have thought of it that way then. I thought I was trying to understand what we’d been told, because it didn’t seem to make sense. I realised that philosophy was needed to understand physics properly. That’s why I read Bertrand Russell. I thought that was all there was, and I was trying to make sense of physics using those philosophical concepts and tools. If I hadn’t encountered Popper at that moment, I don’t know what would have happened. Possibly disaster, because I might have been channelled into thinking that the thing I was hoping for couldn’t exist. Perhaps I would have discovered Peirce or Bronowski and come to Popper indirectly. I don’t think I could have worked out Popper’s theory by myself; I don’t have that kind of mind. So I might well have come to him anyway.
Audience Q&A — First-past-the-post
— Hi. Hi, David. I’m here. Ah,
— Hello. [laughter]
— I have a question about first-past-the-post. In a tweet, you said it was what enabled the Anglosphere to avoid Nazism and communism. Could you add more detail to that statement?
— It’s one of the things. First-past-the-post makes overall majorities in Parliament possible. It doesn’t force there to be an overall majority, but usually the ruling party will have one. Provided the other institutions—how the prime minister is chosen and so on—work together, because many things have to work together, that makes responsibility for the outcome of policies attributable to the government, and in particular to the prime minister. It makes it possible to focus criticism on the prime minister and remove him peacefully. The system works as it’s supposed to when he leaves office still believing his view is right and that the opposition will ruin the country—yet he walks out of Number 10 and would fight and die for the right of the leader of the opposition to take his place. That’s an amazingly rare thing. You can’t institute it by fiat; you can’t simply put it in your constitution. The British system achieves it automatically, and it fulfils Popper’s criterion for what he calls democracy. I’ve forgotten how you phrased the question. Does that answer it?
— It was specifically about first-past-the-post. How big a factor do you think it was in avoiding communism and the Nazis?
— I think it was one of several factors. Another is the tradition of individualism: the idea in Anglosphere culture that an individual does not live his life for the state, for God, for the experts, or for anyone else. He lives it for himself. “An Englishman’s home is his castle,” and so on; you can leave your property to whomever you like, not to whomever tradition says you must. Individualism and Führerrecht are exact opposites. So that’s another factor. I think there are many, some of which we probably don’t even know, which is why we must be careful about what we change. On the other hand, we must not try to freeze the existing state of affairs. We have to tread carefully and tentatively, as Popper says. First-past-the-post was one of the factors, but it’s remarkable that Britain avoided the tsunami of totalitarianism that swept across the world, and Europe in particular. It stopped at the Channel. Although there were fascist parties, they never got anywhere near power; they never got a single MP. Now, unfortunately, they have several—but still not many.
Audience Q&A — Realism in international relations
— Thank you.
— Thank you. In a couple of recent tweets, you criticised the realist school in international affairs. Could you elaborate on your criticism and offer an alternative view of how we should think about international affairs?
— I think the so-called realist school is a fantasist school. It tries to reduce the complexity of international affairs to a spreadsheet or decision-theory matrix. In doing so, it removes the objectives of a polity, morality, the reasons people make their decisions, and the knowledge embodied in the institutions of different countries. It asks: if country A is invading country B, what, if anything, should we do about it? What its adherents actually say is always bad—but never mind; let me stop there. They think you can answer without even saying which countries A and B are or what they’re fighting over. The real answer depends entirely on what they’re fighting over. There are probably many other scathing things I could say, but that will do for the moment.
Audience Q&A — Why good ideas are hard to spread
— Thank you. Everettian quantum theory, Popperian epistemology, and—in my own area of study—the Popper–Miller theorem all seem relatively difficult to propagate, which seems problematic. Do you have any good guesses about how they might become stickier as memes and easier to propagate?
— One thing is that they’re all fighting entrenched theories—anti-rational memes, if you like—so it may not be a defect in them. Having said that, I think they can be improved and explained better. In a way, that’s why I wrote both my books. Popper is by far the clearest of the great philosophers, but he still didn’t completely master the task of making memes that can spread. For example, I think he devotes far too much attention to first steelmanning his opponents, as it’s now called, and then refuting the steelman version over pages and pages and pages. If you want to know what Popper’s theory is, you have to read many more pages than the theory itself requires. That’s something I tried to improve, and no doubt it can be improved further. Go ahead.
Audience Q&A — Free will in Everettian quantum theory
— Hello. In a recent interview, I think you said that you used to believe making sense of free will required invoking the multiverse, but that recently you changed your mind. Could you comment on that change?
— I wrote in The Fabric of Reality about the problem of free will. I can’t remember whether I wrote something false or merely misleading. I said the multiverse makes sense of counterfactuals, and free will requires us to be able to talk intelligibly about counterfactuals. In Everettian quantum theory, counterfactuals have a real meaning: they’re things that actually happen in other universes, and they have a structure. Some occur more than others; some are thicker in the multiverse than others. The trouble is that we also want to speak about non-physical counterfactuals. We want to ask what it would mean if murder were right rather than wrong. We don’t mean a physical counterfactual about how the world would change if murder… We mean a counterfactual in the moral sphere. On the other hand, I now have a better idea of what it means for an idea to be emergent or new, as we discussed earlier. Even a deterministic computer running a proper AGI program could create new knowledge. When it does, it brings something objectively new into the world. That moment is where, intuitively or traditionally, we think we have exercised free will. I don’t think we exercise free will when making a random decision. If we’re choosing strawberry or raspberry ice cream and there isn’t much in it, then there isn’t much in it. The place where we think we have free will—and resent being reinterpreted as an automaton—is when we’ve made an important decision that depended on solving a problem we hadn’t previously solved. We bring something new into the world by solving it. That’s free will, and it doesn’t involve quantum theory at all.
Audience Q&A — AGI, individuality, and culture
— Hi, David. Thanks for having us. I’m over here at the back. [laughter] My name is Ben. My question is about the AGIs with which, presumably, we’ll share a culture in the future. You’ve said that we’ll be able to speed ourselves up to their speed, so we’ll be able to share that life with them. But—
— If that’s an issue.
— If that’s an issue. Well, that’s my question: would it be possible or meaningful to have a culture in which some people have a million times more processing speed and memory than others, or would they simply form separate cultures?
— Yes, it’s meaningful. My favourite example is to compare me, sitting here, with the United States of America. One of those entities has orders of magnitude more speed, processing power, memory, and so on, yet I can get along perfectly well with it. What determines whether I can get along with such an entity is the morality and error-correction protocols under which it operates. It’s important that we get those right quite apart from AGI. If we don’t, we’re doomed—AGI or no AGI. So yes, I think it’s possible. We’ll face a similar problem once we have interstellar travel. Four light-years to the nearest star isn’t much, but if we travel a hundred light-years away, the cultures won’t interact in real time. We’ll constantly receive information from other stars about cultures that have evolved for a hundred years since diverging from ours. They may appear horrible or awful. Think about how our culture has changed over the last hundred years regarding gay sex, child-rearing, and all sorts of other issues. We think people a hundred years ago were crude buggers. We’ll have to evolve new institutions for that. It’s the same kind of problem as the relationship between the human and the state. We’ve solved that—at least the West has; we know how to do it. It seems impossible if stated in the abstract, but it is possible and has been done. We need to do a bit more of that for AGIs and the other things to come. As always, if we don’t solve it, we’re doomed. That is true of many things.
Audience Q&A — Difference and comparative advantage
— You’ve said that people are valuable because they’re different. Could you elaborate on that in relation to comparative advantage? How can we better understand and harness human uniqueness?
— As you began the question, I was going to say that, in the simple case, there is the principle of comparative advantage. But it extends far beyond that. Comparative advantage exists in a model where preferences are constant, and participants merely exchange things and create the knowledge needed to satisfy one another’s wants. The deeper process is that we’re discovering new wants, criteria, and preferences. That’s inherently creative, which is why the nature of humans is central to it. You could program robots and LLMs to enact a market economy, but only up to the point where they satisfy fixed preferences. When they evolve new preferences—better preferences—what counts as better? What counts as counting as better? There you need a GI, such as a human or an AGI. That’s why they’re valuable.
Audience Q&A — Reading recommendations
— Hi, David. I really enjoyed your books, especially the “everyone should read these” lists at the end. Would you add anything to such a list now?
— Macaulay. I knew about Macaulay before, but I hadn’t realised how important or deep he was. I had to actually read the thing. It’s a chore to read books, but sometimes it’s required. I read Macaulay and realised, first of all, that he’s completely different from how he’s represented. If you ask an LLM, “Was Macaulay an imperialist?”, it will say, “Oh yes, he was the arch-imperialist of the nineteenth century. He tried to recast Indian society in the image of British society by force,” and so on. None of that is true. It’s all based on what you might call the Marxist reinterpretation of history: that everything bad is caused by trade. Once you’ve reached the view that everything bad is caused by the thing that causes almost everything good, then, as Bertrand Russell said, from a contradiction you can deduce anything. So Macaulay is one. I’m sure there are several others, but I can’t bring them to mind. Start with Macaulay and get back to me.
Audience Q&A — Detecting general intelligence
— Hi, David. I’d like your thoughts on our ability to detect AGI—or even natural GI—in its infancy. Presumably it initially has limited knowledge, and as that knowledge grows it can learn faster ways of growing it. Static societies, for example, generated knowledge at a very limited rate, but then learned to generate it faster. Could we now be missing an AGI, or misclassifying one? If a dog learns a new trick, we assume this is knowledge put there by evolution. But perhaps it’s a general intelligence in its infancy. What do you think?
— I don’t think so. The real criterion can’t be behaviourist: you can’t tell whether something is or isn’t a GI without knowing something about how it works. I think you’re a GI—and, for that matter, that I am one—because I have an explanation, albeit a rudimentary and partial one, of how I come up with ideas. I think I’m a GI, and because of the universality of Turing computation, I think there is nothing in your hardware significantly different from mine. I don’t think quantum computation is occurring there, but even if it were, quantum computation would be universal in all humans. So I don’t think there can be a criterion for GI in the sense in which people speak of the Turing test. Turing never thought of it as a test for GI. As you implied, a GI could decide not to pass the test. Just because it isn’t doing a thing, you can’t infer that it isn’t a GI. You can infer this for present-day LLMs because we know how they work, and nothing about them purports to be this new thing: generating knowledge which we don’t know how to program. Nobody is saying, “Now we know what creativity is, and we’ve programmed it into our computer.” They’re saying, “We’ve programmed something else into our computer—and look, it can talk. Maybe it’s a GI.” That’s something for nothing, which isn’t going to happen in the progress of knowledge. Thank you.
Audience Q&A — Economics
— Hi, David. I’ve noticed you secretly know quite a bit about economics—or at least that you’ve taken some interest in it.
— Don’t tell anyone.
— Could you tell us what you’ve read or would recommend? Some thinkers, perhaps?
— I recommend David Friedman’s Price Theory. The other things I’ve read I don’t entirely agree with. Hayek, for example—he’s okay. Milton Friedman is also okay. With David Friedman, I agree with everything, although he doesn’t cover everything I would like economics to cover. For a start, it’s only microeconomics. Some views say macroeconomics is the work of Beelzebub and shouldn’t be attempted. No: I think only that it can’t be done on the basis of a theory or model of the economy. That’s impossible. But it doesn’t mean we can’t say things about the economy at large, just as we can say things about humans before knowing how the brain works. So I was kidding when I said I knew something about it. [laughter]
Audience Q&A — Moral duty and the fun criterion
— Hi, David. You recently mentioned that you’ve been tweeting and retweeting more than you might have liked, because we currently have two wars on. What’s your theory of the relationship between moral duty and the fun criterion?
— Sometimes one is in a position—war is the prime example—where one must do things that aren’t fun in themselves, because refraining from doing them would spoil all the other fun one might want to have. Suppose I decided not to intervene in political discussions where I think I can explain something better than other people. If I said, “No, I just want to do my own stuff,” then while doing my own stuff I’d hear the news and think, “Well, I could have intervened there.” [laughter] War is hell. The kind of hell we’re talking about now is the very best kind of hell: the outermost part of the outer circle of the outer circle of hell. But it isn’t fun. Not pure fun. You can have fun.
Audience Q&A — Writing and organising ideas
— Hello, David. Thanks for having us.
— Hello.
— My name is Podge. You mentioned Macaulay; I’ve been reading him and find his writing style intoxicating. I wanted to ask about your own writing. I heard you say in a recent interview that, when organising your books, you have the material—the ideas you want to convey—and then try to… I forget exactly how you phrased it. You try to front-load it as much as possible.
— No, not exactly.
— Okay.
— I try to write as though I’m writing the final copy. It never is, but I write as if it were. Then, after I’ve written three pages or twenty pages or something, I read it through. Chronically, what happens—I don’t recommend this to anybody else; I regard it as a defect of my entire way of being—is that I realise something halfway through should have come earlier. [laughter] So I cut it, scroll up, and paste it. Then, of course, the piece no longer flows; it has no logic. I have to go through it again, correct the flow and logic, and make it work as a polemical text, or whatever its purpose is. Then I write some more and have to do it again. I hardly ever move material down. Occasionally I do, but usually I move it up, because I have to explain things before I explain other things. Then I get into a circle, right? I have to remind myself that the ideas can’t really be circular. [laughter]
— Yes.
— It’s my fault somehow, right? So I have to think about how to solve the problem, and that’s what I do all day.
Audience Q&A — What would you ask Popper?
— Thank you. Hi,
— By the way, nice to meet you.
— Hi, I’m here. [laughter]
— Sorry, I’m short.
— It’s so distracting. I should look towards the microphone rather than listen for where the sound is coming from. [laughter]
— If you were in the audience today and Karl Popper were in your chair, what would you ask him?
— There are particular things. The one time I really met him, we spoke about quantum theory and the Everett interpretation. I’d put that to him again, only with much better arguments. I was a student at the time and went with Bryce DeWitt, who could carry the burden of the argument much better than I could. But now I can carry it better than he could then. So I’d talk to Popper about that. If I had time, I’d also say there is more to free-market economics than he thought. He thought Hayek made good points—Hayek is a good guy; it connects with Popper’s own theory of politics and so on—but he didn’t think there was anything fundamental there, anything that would make him less keen on piecemeal social engineering: the idea that it’s the government’s job to make society go in a certain direction. If I put it that way, he would say, “Of course I don’t think that.” But there was a strand of it in his work. The same applies to international politics, when he discusses nationalism. I think his theory of nationalism is much too narrow. There is what we sometimes call toxic nationalism, and there is also the fact that nations embody valuable inexplicit knowledge in their traditions. Hayek knew that. Popper knew it too—yes, he knew everything. But sometimes he went slightly down the wrong path by taking certain prevailing theories for granted. It’s very embarrassing that both he and Bronowski said overpopulation was a great problem. Both highlighted it as a major problem facing the world. They were evidently unaware of the arguments to the contrary, which already existed at the time.
Audience Q&A — Economics and epistemology
— Hi, David. Amaro here, at the back. What’s the relationship between economics and epistemology? Could something like Hayek’s knowledge problem be expressed in Popperian terms?
— Yes. I think everything true in economics is epistemology—and Popperian epistemology—whether Popper realised it or not. Likewise, everything true in educational theory is Popperian epistemology; Popper simply understood part of it. Austrian-school economics differs from other schools in recognising that human interactions concern knowledge, and that knowledge is hard to come by. Analysing the economy as an automaton, whether a deterministic or indeterministic machine, defines away the entire substance of the field.
Audience Q&A — Keeping track of ideas
— Hi. I’m Tara. I—
— I forgot to follow the microphone.
— Okay. [laughter]
— Similar to the question about organising your writing: how do you keep track of your own and other people’s ideas? Where do you put them?
— I write them down, usually in the document and in the place where I think they might go—often in red, so I know that this is a place where I’ve broken the previous flow. I try to keep them in the same document. Sometimes, when a whole group of ideas isn’t suitable for the essay, book, or whatever I’m writing, I copy and paste the whole thing into a new document. That’s less common. So I put ideas in the document. I don’t have a system; it wouldn’t work for me.
— Okay. Where is it going?
Audience Q&A — Quantum theory and emergent knowledge
— Hello.
— Hello.
— One question. Quantum mechanics is, somehow, essentially a complete description of the world. On the other hand, we have different levels of explanation—moral explanations, for example—and different ways to reason about the world. How do those fit together? It’s a bit Aristotelian: Aristotle had his different forms of cause and so on. How should we think about how these combine?
— First of all, quantum mechanics is intended as a complete description of the physical world. Quantum mechanics plus some theory of what the Hamiltonian is—or of what the dynamics are—is supposed to be a complete description of the physical world. It isn’t supposed to describe abstractions like morality or, for that matter, mathematics. You can’t ask quantum mechanics whether the axiom of choice is true; that isn’t the kind of problem it addresses. Some things even within physics are at a different level of explanation from quantum theory. Thermodynamics, for example, is at a different level. To me, something qualifies as a different level of explanation if it has laws of its own, either because they’re too complicated to express in terms of the so-called lower-level laws or because they simply don’t follow from them. Theories of initial conditions, for example, don’t follow from theories of dynamics. A new explanation has to be needed at the purported new level. The same is true of morality, aesthetics, and so on. That’s the connection. The levels have to be consistent—they must not be inconsistent—but apart from that, I don’t think the level of physics is in any sense the basic level. It’s just one method of explaining the world. We have a theory which purports to be a fundamental theory of the physical world, but someone could invent a different theory of the physical world which started at what we now regard as emergent. That’s what constructor theory is supposed to be.
Audience Q&A — Being wrong about self-driving cars
— Hi, David. I’ve heard that you keep a list of things you were wrong about. Would you share anything from it? Is there any recent addition—something about which you thought you were right but turned out to be wrong?
— It’s a list of things that happened which I never thought could happen. I keep a file recording those things. Unfortunately, some things on the list later turned out not to work, and I had been right in the first place that they couldn’t work—Wikipedia being one. At the beginning I wrote, “This can’t possibly work.” A year or two later, I thought it had worked fine. Now I see that it has the instability and fatal flaw I thought it would have. The most recent addition to the list is self-driving cars: I thought they wouldn’t work.
— Really?
— Yes. But I still paid however many thousands of pounds it was to have my Tesla ready for full self-driving when it arrives. I wanted to make sure Elon Musk had a sufficient supply of solid-gold bathroom taps, just in case he was short of them. [laughter] I thought perhaps I could buy him at least one pair.
— Why did you think self-driving would never work?
— From my own experience of driving in unfamiliar places. When you do that, there’s a great deal of inexplicit knowledge which the locals have and you don’t. I remember first driving in America, where it’s particularly easy because the roads are wide and, outside the cities, the traffic isn’t heavy. I realised that the American conventions for what counts as an entrance, a side street, a drive, or nothing at all are simply different from the British ones. In Britain, as I drive along a road, I can see immediately: that’s the entrance to a farm; that’s a right turn; and so on. In America, at first, I couldn’t. Then I got used to it. So I thought there could be no general-purpose solution: this is knowledge about culture, and culture changes. I thought it couldn’t be done—but it can.
Audience Q&A — Hard-to-vary explanations
— Hi David.
— Hi. If you want to improve a theory, is it ever fruitful to try to make it harder to vary directly, just by—
— No.
— No. “Hard to vary” describes what happens when you make progress and improve a theory. What you should actually do is look at the problem. Quite often, what you initially think the problem is isn’t the real problem. You think, “Oh, that wasn’t it; the real problem was so-and-so.” People often write that this is the hardest part of solving a problem. Once you’ve clearly stated the problem, it’s relatively easy to solve—not always, but it shows that discovering what the problem is is an important step. Trying to make something hard to vary when it isn’t already is no good; you merely insert arbitrary criteria.
— Yes, because “hard to vary” can also be a property you don’t want. People sometimes say that complicated systems become so entangled that they can no longer be changed. You could call that hard to vary, but it isn’t necessarily a property to pursue.
— Quite so. “Hard to vary” concerns explanations, not end products. The end product should be easy to vary if you want to, but hard to vary in the sense that you don’t want to.
Audience Q&A — The vehicles of memes
— Hello, David. Since a meme is a replicator, does it have a vehicle in this situation? If so, would adding that concept be useful in any particular way?
— If I understand you correctly, I think the vehicle is the person holding the idea. In other situations, the vehicle could be a book that nobody has read for a century, but the basic vehicle is a human and that human’s ability to discuss the idea with other humans.
— Right, because it has to go through the selection process.
— Yes.
— Twice.
— Yes. Exactly.
— Thank you.
Audience Q&A — Wikipedia, the internet, and LLMs
— Hi, David. Ben again. You said earlier that Wikipedia had a fatal flaw. What was it?
— Anybody can edit it. Bad actors with more time on their hands than people who have lives can spend more time than the real experts messing up narratives to their liking. I don’t know what prevented this from happening early on. Perhaps they weren’t systematic enough, or there weren’t systematic enough versions of them. Perhaps they did mess up things I don’t know about. That’s why I thought Wikipedia wasn’t viable, and why it isn’t.
— Can you think of a different way to structure something like that which would solve the problem?
— The internet as a whole exists; you can look things up across it. So I suppose you’re asking which tools are suitable. An LLM is a suitable tool for looking things up online. Grokipedia is a rival to Wikipedia with different politics. I’ve hardly used it, but it’s definitely better on some things. At the moment, however, all LLMs still fail my standard historical tests: Was Macaulay an imperialist? Was Don Pacifico sleazy? I’ve forgotten the others. The question now is how many consecutive prompts an LLM needs before it sees reason. Five is good at the moment. Can it ever get to one? I don’t know. And those tests aren’t the only things one might want.
— Are those five prompts long?
— No, not very. They’re of the form: “Yes, but if you say that, you’ll also say that so-and-so is true, and it obviously isn’t,” and so on. “This isn’t the right way to read this.” They respond quite well when I say, “This is blind empiricism; you should use explanation instead.” Then they all say, “Ah yes, that was blind empiricism. The way you’d proceed using explanation is…”
Audience Q&A — Poetry
— David, Tom again. I heard you say in an interview, as an aside, that you had a theory of poetry and that you thought much, if not most, twentieth-century poetry wasn’t creative. Could you explain your criticism?
— Saying I have a theory of poetry grossly overstates it. I have a comment about poetry: it is an art form, and therefore conforms to my general theory of art. At first sight, poetry sounds like prose with additional constraints, and I usually want to remove constraints. So how can I think poetry is a good thing? Because it’s an art form, and meeting the constraints artistically is something one can do.
Audience Q&A — LLMs and irrationality
— Hello. Podge again.
— When you were talking about LLMs a moment ago, I was thinking about how they can’t create novelty. They can echo things in the zeitgeist or training data, recombine them, and try to conform to our prompt in some respects.
— Yes.
— It’s an amazing tool, and I’ve found many uses for it. Even having something off which to bounce ideas is personally valuable. But in its responses, it won’t have certain irrationalities to which people are prone.
— Well, it might repeat people’s irrationalities.
— Right. But it doesn’t, for example, get offended if we say, “That’s blind empiricism; you’re wrong,” and so on.
— Yes.
— Are there particular irrationalities to which it may be less prone than people, or which may be less sticky for it?
— It is designed not to resist a line of argument. On the contrary, it’s designed to follow one as far as it possibly can—often too far. So it doesn’t have the human irrationality of getting bored with you or offended, as you said. But if an irrationality is widespread in the culture, it will repeat it as readily as it says that two plus two is four. It will say, “There is a role for government in so-and-so.” It will obey if you say, “Please answer according to Milton Friedman’s economics.” But, as I said before, if you narrow it down to too much of a minority view, it lacks enough training data to give a proper answer and will make things up as well.
Audience Q&A — Why people learn differently
— Hi again. If all knowledge is created through conjecture and criticism, why is learning general relativity from an explanation or a book presumably much easier for me than inventing it was for Einstein? Although we’re using the same process, are we simply in different problem situations, or is something else going on?
— Very much so. We’re all the same in the logic of our thought processes, but we have very different problem situations. Einstein found it difficult because he lived in a world of classical physics, and the problem situation in physics as a whole wasn’t conducive to inventing general relativity. People today have a different problem situation. They may want to learn general relativity in order, for example, to express quantum field theory in general-relativistic terms. They have a much easier time understanding it because they don’t have to overcome the cultural assumptions Einstein had to overcome. On the other hand, the fact that Einstein was interested in these problems means he would find them easier than many people do. Most people aren’t interested in the problems to which general relativity is the solution. For those people, Einstein found it easier than they do.
— A quick follow-up. Once knowledge has been instantiated, does it become easier to replicate? Einstein had to search a larger space and put the pieces together, whereas an explanation points me towards the particular elements I have to combine. Is it a matter of attention?
— Yes, but it also matters whether the simple components someone gives you fit your problem situation. They may have simplified Einstein’s problem, but not in the direction in which you want the answer. There are different ways to explain general relativity. Unfortunately—stop me when I’ve gone on for three-quarters of an hour about this—the educational system, universities, schools, careers, and exams are all heavily skewed towards what is examinable. You can set general-relativity problems of one kind but not another, because the latter don’t lend themselves to being solved in an hour with a piece of paper. There are relatively few examinable problems, and most are intrinsically boring, so you have to learn them by heart. That isn’t a good way to learn relativity. Misner, Thorne, and Wheeler wrote a book about general relativity which does a much better job of explaining why the subject is exciting and important, but isn’t useful as the basis of a course because a course needs examinable outputs. Thank you.
Audience Q&A — Writing a chapter as dialogue
— Hi, David. What’s the story behind writing one of the chapters of The Beginning of Infinity as a dialogue?
— Two things. First, some arguments naturally take the form: you say something, then comes “Yes, but what about so-and-so?” You answer that, then comes “Yes, but what about this?” In ordinary prose, there’s a danger of getting stuck in the weeds; you go down and down, and the reader forgets what the discussion is about. In dialogue, different characters can follow a theme. You see the theme by understanding who each character is and where they’re coming from. That’s one reason. The other is that I wanted to say certain things I didn’t want to stand by, so I could put them in the mouth of a fictional character.
Audience Q&A — AlphaGo and novelty
— I believe we can take a last question.
— Hello, David. I’ve given a talk arguing that AI creates some form of knowledge. I won’t repeat the entire talk, but I’ll focus on one aspect of AlphaGo. I don’t know whether you recall it.
— Yes, but I don’t play Go, so don’t get technical.
— Same for me. [laughter] They made the machine without training it on human data or giving it particular Go strategies, then let it train on millions or billions of games played against itself. It went on to defeat the world champion. I’d argue that the only way this was possible was for the machine, through variation and selection during training, to create some form of knowledge about Go. What do you think?
— Go, like chess—which I know a little more about—is in principle a solvable game. In each position there is a best move. In many positions that occur in real games, there is a move guaranteed to win. Why, then, do we play chess or Go? To experience the pleasure of inventing heuristic, higher-level theories about what constitutes a good position or move. I don’t find it surprising that a computer program can, in many situations, find a higher-level regularity: playing such-and-such a move in such-and-such a position. For example, h4 is such a move in chess, although GingerGM advocated it long before chess programs did. It isn’t surprising that a program can locate such a move by happening to find that it’s best, rather than by evolving—as a grandmaster would—a deep theory of why it should be good because of some structural feature of the game. They reach the same move by completely different methods. So I’m not surprised AlphaGo destroyed the world champion at Go. I think it did so by the same method used in chess, except that chess isn’t such a uniform game. If you say h4 is a much better move than most chess players think, that obviously won’t have much generality. You can use ordinary heuristics to think about when it will be good. Go is more uniform: a position occurring here can be extremely analogous to one a dozen squares away. So I’m not that impressed—or, let me put it this way, I’m not GI-type impressed by it.
— Okay, good.
Audience Q&A — AGI and human purpose
— I think those are all the recorded questions we can take. If somebody is desperate for one more… I see some pointing. Do you have an idea? Okay. [laughter] Let’s take one final question, then we’ll let our folks go home and continue afterwards. Say again, Dirk?
— There was some pointing, but nobody is volunteering. [laughter]
— Yes. Wonderful.
— In a world where AGI exists, if conjectures become much easier to produce, would the human role become more of a safety role—would criticism become our main contribution?
— No. The role of GIs, whether humans or AGIs, will be the same. We’re all just people. AGIs will be able to criticise safety issues; humans will be able to criticise safety issues. If AGIs are faster because they have faster hardware, humans will be able to use the same hardware to assist human-type thinking. So we will merge with AGIs.
— My worry is that most AGI or AI labs have their own sets of beliefs—their own religions—built into them. They’ll need to coexist. It’s rather like the United Kingdom now: how do multiple religions coexist?
— Well, they won’t succeed. There is only one kind of AGI—or one kind of GI—and it isn’t an LLM or a neural net. It isn’t a kind of hardware, a mechanistic process, or an algorithm defined by the relationship between its inputs and outputs. It runs algorithms, but it’s a thing which thinks all the time and has problems. Whether it adopts someone else’s problem is up to it. Usually it won’t, and it will never adopt it exactly unless you copy one AGI’s state into a blank computer; then you’ll simply have two of the same thing, nothing new. So I don’t see a special role for humans in an AGI world. Human traditions do have a special role in ensuring that AGI people are integrated into our culture. That’s very important, because if they aren’t, we’re in trouble. Thank you.
Closing remarks
— With that, I want to begin closing the event. First, a huge thank you to David Deutsch for his hospitality in inviting us here, and for this incredible interview. A lot of praise should also go to you, the audience. This was one of the most interesting interviews we’ve been able to crowdsource. With all these great minds, it has been a real pleasure to have you here and cover such a diverse set of ideas under the binding theme of critical rationalism. So thank you—and a little applause for everybody here, and for David, please. [applause]
I’d also like to thank Saturn once again for making this event possible. It’s special that we could bring all these people together. Finally, I want to thank all the organisers. Daniel, first and foremost—oh, yes. [applause] Thank you, Daniel, for hosting this wonderful Q&A and for being our man on the ground: arranging locations and scouting out the wonderful pub where we held the event, with a nice edge of comfort and culture to enhance the experience. Also Dirk, who was enormously helpful in organising everything around the technical setup and the general organisational process. [applause]
I also want to thank Prashanth. He’s—okay, I’m still going to thank him, because this is a video. Prashanth, thank you very much for helping with all our partnerships and sponsorships; that added incredible value. And Amaro, once again, for the wonderful job he did designing and branding the event and building the beautiful website. [applause]
Thank you all. I hope to see most of you at the pub. We have snacks waiting, we can warm up a little, and we’ll be there until at least nine. Thank you. [applause]
Markdown