2022-04-24 The Joe Walker Podcast #139 - Against Bayesianism
Duration: 01:39:02
Transcript
Joe Walker
You’re listening to the Jolly Swagmen podcast. Here’s your host, Joe Walker. Ladies and gentlemen, boys and girls, swagmen and swagettes, welcome back to the show. It has been a while. I feel like I was saying that only a little over six months ago after my last hiatus. As with the last hiatus, work responsibilities have kept me from podcasting up a storm, but it’s wonderful to now be back in the saddle. Running and hosting this podcast is a singular privilege and so much fun, as I think you’re going to be able to tell from the next few conversations I will be releasing. This episode represents the beginning of season six of the show and our first episode of 2022. This year, my goal is to publish 20 timeless podcast episodes, maybe more. We shall see. Some housekeeping items before I introduce this episode. First, to everyone who has emailed or Twitter DMed me while I’ve been on break, thank you for your patience. I hope to reward it. Second, there has still been some great activity on the podcast front, even though I haven’t been publishing episodes recently. I have a revamped website and a huge thanks is owed to the wonderful Pete Hartree for this. You can view the website at thejspod.com and there you can view episodes, transcripts, and sign up for my weekend newsletter, which has continued going out every weekend, even during my podcast hiatuses. I’ve also set up a new studio for in-person podcast recordings. I recorded my first conversation there on the 9th of April. You can check out pictures of the studio on Twitter. My handle is at Joseph N. Walker. Finally, and as always, I love when you write in with ideas or to discuss episodes or things I’ve shared in my weekend newsletter. I’ve really enjoyed hearing from listeners over the past six months and many such interactions have turned into real life catch-ups, projects, and partnerships. I’m very lucky to have such a smart, thoughtful, and open-minded audience. And another one of my goals this year is to try to meet more of you in person, including through some live events as well. If you ever want to get in touch with me, you can reach me at joe at thejspod.com. Let me introduce this episode, which I’ve titled Against Bayesianism. This episode is basically about prediction, what kinds of predictions are illegitimate, what kinds are legitimate, and where Bayesian updating is appropriate. This is not an episode about Bayes’ theorem, a wonderful tool that I use intuitively almost every day. The theorem, named after one of its progenitors, the Reverend Thomas Bayes, I say one of because Laplace actually discovered it independently and developed it into the form we use today, tells you how to update a prior belief in light of new information. It’s straightforward and fairly well known. It goes like this.
Joe Walker
The probability of A given B is equal to the probability of B given A multiplied by the probability of A over the probability of B. This episode is not about that rule. Rather, it’s about taking up that rule as a hammer and seeing too many things, including scientific theories as nails. It’s a podcast about Bayesianism. Applied to scientific theories, Bayesianism holds that, in the words of my guest, rational credences, that is degrees of belief, obey the probability calculus and that science is a process of finding theories with high rational credences, given the observations. Applied to decision-making, Bayesianism is, to quote Ken Binmore, the doctrine that Bayesian decision theory is always rational. Whether in philosophy or macroeconomics, science or decision-making, Bayes’ rule rules. But in this episode, my guest attempts to dethrone Bayesianism. My guest is David Deutsch, a British physicist at the University of Oxford. David is widely regarded as the father of quantum computing. He’s also the author of two books, The Fabric of Reality, published in 1997, and The Beginning of Infinity, published in 2011. They’re extraordinary, zany, brilliant books. They’re popular science books, but they’re more than that. If you haven’t read them, they would be close to the top of my reading list for a generalist slash optimist interested in the world of ideas and in improving the real world. Both are remarkable for the fact that, as many others have observed, they make original contributions to epistemology while being popular science books. It might seem like a quaint indulgence to be discussing epistemology in the philosophy of science at a time when Russia is visiting destruction upon Ukraine, and the prospect of a great power conflict feels more salient than at any point in the last few decades. But this podcast is really about progress and how to create it. Progress, scientific, technological, economic, may seem like an obvious good, but there is a growing counterculture hostile to it. I’m publishing this conversation now precisely because of, not despite, the horrors of current affairs. Because I’ve never been more convinced that progress is the only thing keeping us from stumbling sooner or later into the abyss. I hope you enjoy the conversation. David Deutsch, welcome to the podcast.
David Deutsch
Thanks for having me.
Joe Walker
I wanted to focus in particular today on prediction and the types of predictions that we can make legitimately. I’d like to begin with a bit of context. I was hoping you could tell me about Charles Parkin and how he led you to Karl Popper.
David Deutsch
Oh, Charlie Parkin. Yes. Well, he was my tutor, as they call him in Cambridge, which is misleading because tutors do anything except teach. I think in Oxford, they used to call them moral tutors, but even that is misleading. It’s basically a member of the college whose job it is to connect with the particular students that they’re assigned to and be on their side. If they have some problem with the bureaucracy or with the university or with the college, then if you’re in trouble or just if you need something, you would go to the tutor. There was a thing that you’re supposed to go and see your tutor at the beginning and end of each term to check in with them. They check that you’re not on drugs and stuff like that. I never had anything to say to my tutor. He was a historian, which is not now, but was then pretty far from my interests. On one occasion, I had written this essay just for myself because I’d read Bertrand Russell’s History of Western Philosophy and another book. I was really taken with this idea of philosophy of science. I thought, I want to be a physicist and I want to do it right. You know, so I wrote this article about how it’s important to do induction right, as it says in Bertrand Russell. I sent this to Charlie Parkin so that we would have something to talk about when I went to see him. I remember this very well. He said, um, hmm, induction. Hasn’t that been proved wrong by that Popper chappie? Well, I’d only heard of Popper once before, which was from my mathematics teacher in school, but I’d never followed up anything. I said, oh, well, you know, I didn’t know. Um, he said induction is old hat now. So I, uh, I thought, well, if it’s old hat, I need to know what new hat is. So I went out and bought a Popper book. Um, and that was the beginning of, beginning of, um, a complete change of course in my philosophical life.
Joe Walker
And I believe you actually met Popper, right? Can you tell me what that was like?
David Deutsch
Yes. Uh, I met Popper once, unless you count one of his lectures that I went to, um, uh, where I didn’t meet him personally. So I only met him personally once. I met, um, I went to his home with, um, my, my boss, Bryce DeWitt, who also wanted to meet Popper. And, um, uh, we went to his home and, uh, we mainly wanted to tell him that, um, although his philosophy was, you know, uh, amazing groundbreaking, uh, he, he’d got, um, quantum theory completely wrong. And he hadn’t even understood what the problem was, let alone what the solution was. And the solution was, uh, Everett’s multiverse interpretation or so-called interpretation. It’s a Everettian quantum theory as we now prefer to call it. Um, and, uh, he just rejected that out of hand again for rather silly reasons. So we, we, uh, we discussed many things with him, but, among other things, we got around to discussing Everett and we explained it to him, you know, what the problem really was and what he got wrong. And he listened incredibly carefully, asked all the right questions. Uh, you know, we’d been told he’s intellectually very arrogant and he, he, uh, shouts down opposition and so on. None of that. We, we found him incredibly, you know, Popperian in his, in his attitude to ideas. And, uh, at the end, he said, uh, well, uh, and I’ve got a book in print now and I’m going to have to change one of the chapters, something like that. Uh, I’m going to have to make a radical change to something. I forget what it was. And, uh, we thought, wow, you know, we’ve, we’ve kind of succeeded. But then when that book came out and I forget actually which book it was, he wrote several that mentioned quantum theory, but it had none of that in it. It, it was all back into his original view. So I guess he must have changed his mind. Uh, whereas in the heat of the moment, he’d, he thought we had a point, but then thought, no, no, they don’t have a point. So, um, that was my, my one and only meeting with Popper.
Joe Walker
Yeah. Did he show any engagement with the idea or did he just completely dismiss it?
David Deutsch
Yes. Oh no. In the book that he subsequently published. No, no. Uh, as far as I remember, anyway, the only mentions are asides and they were rather disparaging.
Joe Walker
Right. So why was encountering Popper such a pivotal moment in your intellectual development?
David Deutsch
Um, it’s hard to, um, it’s hard to express exactly why, but, um, for me personally, psychologically, I had, uh, my idea of philosophy and the philosophy of science and what philosophy is for and what it can do and so on. Um, like when I was in school and an undergraduate at first, uh, it was really the everyday view of philosophy. And when I read, um, Russell, it was again the common sense view because induction is common sense. You know, if we see the sunrise every day, then we think it’s going to rise the following day as well. You know, there is that, that kind of thing. But if we see it not rise, then we know that there’s something wrong with our theory that it’ll rise every day. Uh, and that’s what I thought. That’s what Russell thought. And then when I read Popper, I saw that not only was that wrong, but it took philosophy itself to a whole new level. It’s, it’s like, this is seriously, this is, uh, serious, um, thinking about what the truth of the, of the matter really is. And so it was the seriousness of Popper which first got me rather than the content. And it took me actually several years. I mean, I’ve been trying to think back from time to time how long it took me before the time when I would say, yes, I’m a Popperian to the time when I actually got it. I think it was about four years and several more Popper books.
Joe Walker
When you say the seriousness of Popper, what makes someone serious in that respect?
David Deutsch
Uh, well, it’s, uh, again, it’s very hard to describe in words, but it’s, it’s following ideas through, um, and insisting that things make sense. So the, uh, the trouble with the, uh, theory of induction is that if you follow through any strand of like, how can this be, you end up with a problem, namely that it doesn’t make sense that the philosophers call this the problem of induction. Uh, like the original problem of induction was, uh, um, we, we see all these instances of things like sunrises and we, we infer that the sun is going to rise again, but that inference is not logically valid. So logic had been developed to quite a high degree already in antiquity by Aristotle and people. And then even Aristotle already realized that this kind of inference is just not a valid inference. It doesn’t follow. And, uh, then, you know, people tried various ideas. Okay. Maybe it’s not logically valid, but there’s another form of reasoning, which you can call inductive reasoning. And that somehow makes sense. And every attempt to make that make sense didn’t work either. And then, uh, then I realized that, you know, this whole problem is a misconception because it’s just not true that the future is like the past. Um, you know, the, there’s one thing about the future, and this may come into prediction, if you want to talk about that later, um, is that it’s not like the past. It’s never like the past. And then, well, the inductivists might say, well, yeah, but it’s like the past in some ways. And it’s, it’s different from the past in other ways. So it’s approximately like the past. And, you know, so none of that works. And what Popper did was he said, okay, uh, what are the assumptions that lead to this so-called problem, i.e. this so-called refutation, sorry, this, this refutation of the whole theory? What are the, um, assumptions behind it? Which one of which, one or more of which must be false. And, uh, so he took seriously that there is something wrong with our theory of not our theory of just scientific knowledge, but our theory of what knowledge is. And, uh, knowledge had traditionally been thought again, since antiquity, been thought of as what was later called justified true belief. So it’s a kind of belief. Knowledge is a kind of belief and it’s true and, uh, it is justified. Then you can also modify that by saying it’s, uh, it’s a form of belief that is, is mostly true, or it’s a form of belief that’s probably true or probably justified or partially justified, you know, where everything’s been tried along those lines. And, um, Popper realized that the argument, the problem of induction actually implies that there’s no such thing as justified knowledge in the first place and that we do not need knowledge to be justified in order to use it.
David Deutsch
Um, and, uh, there is no process of justifying, uh, a theory. So theories, according to Popper, are always conjecture and thinking about theories is always criticism. It’s never a justificatory process. It’s, it’s always a critical process. So, uh, as David Miller says, uh, a theory doesn’t need to have any special credentials to be allowed into science. It’s, it’s a, it’s a conjecture. Any conjecture is allowed into science, but once it’s into, into science, it’s then criticized. And when it’s criticized, uh, successfully, it is dropped. And, uh, so that’s, that’s the, that’s the, that’s where Popper begins. Then he, he has to answer all the questions, you know, why, what’s the rational reason for acting on theories and so on. But once you’ve got the idea that you don’t need justification, everything eventually falls into place and it falls into place in a structure that makes sense and it makes sense of science and even beyond science.
Joe Walker
So his solution to the problem of induction was to sort of reframe it and say that justification isn’t needed in the first instance. For people wanting to read a good summary of his solution to the problem of induction, would you agree that the first chapter of his book, Objective Knowledge is probably the best place to go?
David Deutsch
Yes. Many people say that. I, um, I think it, where you should start with Popper rather depends on where you’re coming from. Uh, because his, uh, what I’ve described is, is, is his philosophy of science narrowly conceived, but he had a very broad attack on different areas of philosophy. Basically all the same thing. It’s, it’s all the idea of thinking of starting with problems rather than starting with, um, you know, starting with existing theories and criticizing them, rather than seeking justifications for theories. Um, some people come to Popper via his political philosophy, though he denied being a political philosopher. Uh, but he, but he was, and he was the, you know, the greatest so far. Um, uh, so some people, some people, um, actually back in Cambridge, it was very hard to find Popper books in bookshops at the time, and, uh, there was no internet. And, uh, so actually the first one I read was, um, the open society and its enemies volume two. So, you know, that’s the only one I could find at first. And then I, then I found volume one and read that too. And that, that had the only direct connection that that had with the philosophy of science was the, in the underlying approach. And that, that’s what attracted me. And that’s why I looked further and tried to, you know, whenever I went into a bookshop, I, um, first looked for Popper books in the philosophy section, very rarely found one, but I, yeah, I think, um, objective knowledge is a good place to start or conjectures and refutations. Also other people find, uh, interesting. Um, uh, so yeah. Uh, but it, it, as I say, it depends where you’re coming from.
Joe Walker
Why do you think his books were so conspicuously absent from the shelves of, um, Cambridge bookshops?
David Deutsch
I don’t know. There is a mystery about Popper’s like reception in the academic world. Um, uh, which I don’t know about in that that’s kind of the history of ideas, which, uh, I’m not an expert on. And I’d rather, I’m more interested in ideas than in the history of ideas. Sometimes the one is needed for understanding the other, but, um, uh, I know that Popper had great difficulty, especially with Oxford and Cambridge, but with the academic world generally, and he would never have come to England as I understand it. If it hadn’t been for, um, Hayek, who was professor at the LSE, um, kind of, uh, causing the LSE to create a professorship just for Popper, um, in what I think was called the methodology of science, the scientific method, something like that. And his, his, uh, lectures famously began and you, you can find his first few lectures on the internet. The first one begins something like, um, uh, I want to, uh, I want to warn you that although I am called a professor of, uh, scientific methodology, and I’m the only one in the British empire, as he put it at the time, um, uh, there is no such subject. There is no such thing as scientific methodology and so on. And he goes on, you know, brilliantly from there on.
Joe Walker
That’s great. I’ve heard you recommend the myth of the framework as the best of his books, maybe not for beginners, but certainly for people who’ve already read a few of his books. Why do you recommend the myth of the framework in particular?
David Deutsch
Yeah, well, it’s, it’s not the book. So many of his books are collections of essays or lectures or whatever. And the myth of the framework, the book is such a collection. And what I always recommend is the particular essay within that book called the myth of the framework. So the book is named after that one essay or that one chapter. Um, I think it’s brilliant because it, this, uh, reaches out beyond philosophy of science, philosophy of politics to just a general, uh, attack on, I don’t know what you would call it, relativism, uh, including post-modernism and all sorts of bad ideas about ideas, uh, where the framework, the actual myth of the framework that he criticizes is that for two people to make progress in a discussion, it is important that they have an area of agreement and that they locate that and then work out from there to create agreement. Now, Popper attacks that idea from all directions. First of all, he says, the discussions can be valuable and usually are even if you never reach agreement. And this, this is, uh, I think a crucial idea of Popper’s because, uh, again, this idea that, that, um, the objective of a discussion is to reach agreement is authoritarian. It’s the idea is that you’re, you’re creating together a kind of authority, a kind of uniformity, um, whereas in fact, all we have is conjectures and we are going to be wrong in various ways and we’re never going to arrive at the final truth because there’s always improvements to be made. And when you, um, when you have a public controversy, like people say, debates in parliament are useless because nobody ever changes their mind. Well, first of all, people do sometimes change their mind, but that is not the point. The point is that by having a debate, you improve your, not your, you don’t improve your agreement necessarily with the other side, but you improve your understanding of the other side. You can, if you are right, you improve your own arguments so as to be better. And, uh, if you think about real life, you know, how people, people changing their minds about things, you can very rarely remember a case where somebody has changed their mind during a debate. And yet, if you look at the big picture, if you look at opinion polls about, you know, would you live next to a person of the, of a different race? You know, a generation ago, it’s like 20% of people would, and now 95% of people would. And in that time, you can’t find anybody who says, oh, right, now I’ve changed my mind about that, or hardly anybody. What has happened is that they changed their view, they changed their view on a larger scale and on a deeper scale, including in the first instance, the type of reasons that they give themselves for their ideas. So as somebody, I can’t find this quote, but there’s a marvelous quote by some moral philosopher saying, the reason we need moral philosophy is that people change the reasons for their behavior before they change their behavior. That, you know, you justify it in a different way.
David Deutsch
Now that you’re justifying in a different and better way, even though you’re justifying the same view and the same behavior as before, you’re now changing. And eventually, that will lead to your change, changing your actual behavior in that, in that little way we were talking about. But you never see that because it happens as a result of a deeper shift. Um, so this in practice is what happens. And in theory, the theory of it is, is, was first understood, I think, by Popper, uh, and expressed in that essay.
Joe Walker
That, that quote you shared of the moral philosopher also reminds me of that great quote by John Stuart Mill. He who knows only his own side of the case knows little of that. Right. How do you define Bayesianism and why in your view is Bayesianism a form of inductivism?
David Deutsch
Right. Well, um, the word Bayesianism is used for a variety of things, uh, a whole spectrum of things at one end of which, uh, I have no quarrel with whatsoever. Uh, and at the other end of which, I think is just plain inductivism. So at the good end, Bayesianism is, uh, just a word for, um, using, uh, conditional probabilities correctly. So if you want to know, you know, uh, you, you find that your, uh, your milkman, uh, was born in the same small village as you and, uh, you know, you, you, you’re wondering what kind of a, um, a coincidence that is, and so on, you, you’ve got to, uh, look at the conditional probabilities like, uh, rather than the absolute probabilities. So there, there, there isn’t just one chance in so many million, but there’s a smaller chance. And then what, um, against what background of, uh, population are you taking this estimate of the chance and so on. So, you know, you, if you’re not careful, you can end up, uh, concluding that your milkman is actually stalking you. And that’s because you’ve used probability wrongly. So that, that is one end of the, of the spectrum, which I have no quarrel with whatsoever. At the other end of the spectrum, a thing which is called Bayesianism is what I prefer to call Bayesian epistemology, because it’s the epistemology that’s wrong. Not the, not Bayes’ theorem. Bayes’ theorem is true enough, but Bayesian epistemology is just the name of a mistake. It’s, it’s, um, it’s a species of inductivism and currently the most popular species. Um, so the idea of Bayesian epistemology is that first of all, it, it completely swallows the justified true belief, um, theory of knowledge. So it’s saying, how do we increase our knowledge? Well, we increase, uh, we, we increase our knowledge whenever we increase our credence for true theories. Credence is belief. Belief is, according to Bayesian epistemology, um, measured by a measure that is basically a probability. In fact, all probabilities are supposed to be these beliefs, um, which is another mistake, but never mind that for a moment. Uh, so, um, uh, so the idea of science and of thinking generally in Bayesian epistemology is that we’re trying to increase our credence for true beliefs and decrease our credence for false beliefs. And so they use Bayes’s theorem to show that when you encounter a true instance of a general theory, your, uh, and you use Bayes’s theorem to calculate the new probability of that theory, the new credence for that theory, it has gone up. And so the basic plan of Bayesian epistemology is that that is how credences grow, go up. Um, and the way they go down is if you find a counter example. So credences of theories go up when you find a confirming instance and down when you find a disconfirming instance. And that just is inductivism. That, that is another way it, it’s, it’s inductivism with a particular measure of how strongly you believe a theory and with a particular kind of framework for how you justify theories.
David Deutsch
You justify theories by finding confirming instances. Um, so, uh, that is a mistake because although it’s true that the credence of a theory, so if theories had probabilities, which they don’t, but if theories had probabilities, then the probability of a theory, probability or credence in this, in this philosophy, you, they’re identical, they’re synonymous. Um, if you find a confirming instance, the reason your credence goes up is because, um, because, um, some of the theories that you, that were previously consistent with the evidence are now ruled out. And so there’s a deductive part of the, of the theory that, that, uh, whose credence goes up. But if you, if you, if you ask, apart from the deductions you can make, because the instances never imply the theory. So you want to ask the part of the theory that’s not implied logically by the evidence, why does our credence for that go up? Well, unfortunately it goes down and that’s the thing that Popper and Miller proved, um, in the 1980s. And, uh, I, I’ve been, I and a colleague, um, Matjaž Leonardis, um, have been trying to write a paper about this for several years to explain, um, why this is so in more understandable terms. Unfortunately, Popper and Miller’s, uh, two papers on this are very, uh, condensed and mathematical. And, uh, they, they use, uh, they use, uh, kind of a special terminology that they made up in order to prove this. So the paper hasn’t been taken on board and we would like it to be taken on board, but we haven’t yet managed to solve the problem, which evidently they didn’t, of how to present this.
Joe Walker
I’m curious, are you aware of some of the analogous critiques made of Bayesian decision theory by …
David Deutsch
People like Ken Binmore? Uh, no. Um, I, I’m not aware of having any quarrel with Bayesian decision theory, except unless this is referring to its ambiguity, uh, that, that, that you never know, uh, um, rather like the Duhem-Quine ambiguity in scientific reasoning. If that’s what you’re referring to, then I do know about it, but I haven’t specifically read about it.
Joe Walker
No. So I think the best place to start would be Binmore’s book, Rational Decisions, but in the book he takes Bayesianism to mean the doctrine that Bayesian decision theory is always rational and he builds on Leonard Jimmie Savage’s distinction between small and large worlds. So small worlds are worlds where you can, you know, look before you leap. Large worlds are worlds where you have to cross that bridge when you come to it, so to speak. And Savage argued, although everyone in, you know, say economics, macroeconomists seem to have forgotten this, that Bayesian decision theory is only applicable, like it’s only sensible in small
David Deutsch
Worlds? Within large worlds, do you mean, um, worlds where there’s a finite number of things that you have propositions about, therefore when you find that one of them is true, you’ve actually made inroads into the whole set, whereas for infinite things you never make any inroad into …
Joe Walker
The whole set. Exactly, exactly. And I guess archetypal examples of large worlds are like high finance, the macro economy, et cetera. And it just doesn’t make sense to apply Bayesian decision theory in large worlds. So Binmore has this long kind of rant against Bayesian decision theory, but he argues that Bayesians are acting as if they’ve solved the problem of scientific induction, even if they don’t explicitly acknowledge that. I agree that they are, and I agree that that’s an error. So why is the future of civilization unpredictable in principle?
David Deutsch
Um, because future knowledge, because it’s going to be affected by future knowledge, and future knowledge is unpredictable. So, uh, if you, if you think, well, the example I give, in my book, um, is, is that, uh, if you’d been trying to predict the future of, um, of, um, energy production in 1900, um, you wouldn’t have included nuclear energy because, uh, radioactivity had only just been discovered in 1900 and it wasn’t known that it could be used to produce energy. So then, um, and there was no way of predicting that it was going, nuclear energy was going to be discovered because if you had predicted that, that would be equivalent to already predicting it in 1900. So that’s a logical contradiction. You, you, you can’t know knowledge that you don’t know. So, uh, now suppose you’d been magically told that there would be, uh, nuclear energy, then you, you might’ve predicted that, uh, okay, carbon dioxide is not going to build up in the atmosphere, um, uh, because we will have, by the mid of the, middle of the, 20th century, we’re going to have nuclear power and we won’t have any use for fossil fuels or, you know, much less use for fossil fuels anymore. So there won’t be global warming. And so you, you could predict that there won’t be global warming on the basis of the knowledge in, best knowledge known in 1900, but it was not known in 1900 that in the mid 20th century, there would be an environmental movement that would stigmatize nuclear energy and so on. Uh, and so at each stage, you don’t know what the future of knowledge and these examples illustrate that knowledge can be erroneous as well. So that’s another thing that Popper took seriously, that false theories also contain knowledge. And, uh, so this is, um, uh, this is a nice example to show that it’s impossible to know the future that’s going to be affected by knowledge. Now, is the future, you know, which parts of the future are going to be affected by knowledge or not? Well, that’s also unknowable. So, uh, you know, we, we predict the orbit of Mars, but the orbit of Mars is, is only going to be, our prediction is only going to be correct if nothing intervenes. So human knowledge could intervene. We, we might create the knowledge to shift Mars and we might want to shift Mars for some reason or another. And in the next hundred years or thousand years or million years, we might want to do that. And whether we do it or not depends on the knowledge we create and that knowledge, not just scientific knowledge, but also all other forms of knowledge that moral knowledge, political knowledge, aesthetic knowledge, all those might, um, affect the orbit of Mars in the future. So, but that doesn’t mean that it’s completely useless because, uh, to, uh, try and predict anything, um, because we have explanations.
David Deutsch
We have experts. So we, we, what I’ve just said is not just a conditional prediction. It’s also an explanation because I’ve have been saying that it would need some, um, extreme changes in the human condition for human knowledge to affect Mars. Whereas if you talk about human knowledge affecting, let’s say the atmosphere, um, then there’s, uh, the best explanatory theories we have are already that human knowledge is affecting the atmosphere now and will affect it more in the future. Now that might be false. Both those things might be false, but every particular theory about how it can be false is subject to criticism and has failed criticism. So our, our best explanatory theories about the future say that, um, the atmosphere is being affected and will be affected by more in the future. And we, um, we can therefore conclude that given what we want from the atmosphere, we would, we would do best to create the knowledge to make it change in the way we want.
Joe Walker
So is the key point of differentiation between legitimate prediction and illegitimate prophecy that legitimate predictions rely on good explanations?
David Deutsch
Exactly. And, but their legitimate predictions are not justified knowledge. They are, they are conjectures just like everything else. It’s just that their rivals have failed criticism, which doesn’t mean they’re false. They have just failed criticism. And the rational way of proceeding is to proceed according to the best explanation.
Joe Walker
And they are of course subject to disappointment. Yes. So for most of our species history, knowledge was sparse and grew slowly, if at all. Yes. Does this mean that, that people could have made better predictions about the near futures of their societies than we can of ours? For example, would it have been easier for someone in Ming China to make predictions about the future of their civilization than it is for a 21st century American to make predictions about the future of the United States? Yes. How could people in the past have known ex ante that their seemingly static circumstances endowed them with this predictive power without extrapolating their circumstances forward?
David Deutsch
The assumption was, it’s not just any old prediction. The assumption was that nothing would change. Now sooner or later that assumption is going to be proved false. In almost all cases, it was proved false by the destruction of that society, that civilization. Almost all civilizations that have ever existed have been static until they were destroyed, either by nature or by other humans. So whether you call that reliability of prediction or not, it is a matter of taste. You know, if you’re in an empire that’s, that in fact is going to last for 450 years and you’re at the 200 year mark and you predict, well, nothing’s going to ever change, then you can predict that, you know, the kind of diseases that you have now are still going to be experienced by your great-great grandchildren. Until you’re wrong, you know, until it comes to the 400 year mark and your great-great grandchildren are going to have it much, much worse than you or much, much better than you. So it depends how you, how you calibrate predictability. In a static society, you can make predictions conditional on the survival of the static society, whether or not you know that you’re making them conditionally. In a dynamic society, it’s much easier to see that your predictions, it’s the other way around. Given that your society is going to survive, the future is opaque.
Joe Walker
So to come back to good explanations, there are some quotes in The Beginning of Infinity that could potentially be misconstrued as prophecies. So I just wanted to give you the opportunity to clarify them. So for example, on page 455 of the paperback, you talk about how humans will achieve immortality within the next few lifetimes. And I imagine that someone might pounce on that and say, ah, David’s contradicting himself. That’s a prophecy. But I suppose your retort would be there’s like a good explanation underlying that claim. Is that fair to say?
David Deutsch
I can’t remember the wording. It’s perfectly possible that I worded it in a way that was either ambiguous or plain wrong. So if I said humans are going to solve this problem within X years, then that is a prophecy and I shouldn’t have put it that way. Now, if I said I expect this to happen, then that technically escapes the criticism of prophecy. But it depends on the context. If I expect it can be taken in context to mean I predict, then it is a mistake and I shouldn’t have said it. But if it’s a description of my personal conjecture of what’s going to happen, then it’s accurate. And I think it’s so. It is based on an explanation, but the explanation is kind of in a negative form. At present, I see nothing in our existing best theories of biology that suggest that there’s a law of physics that says that lifetime has to have a particular finite limit. We know that there are organisms that don’t have that limit, like most microorganisms and so on. And the processes that we know that kill people are all of the form something goes wrong physically, which we can see and which we could undo if we had the knowledge. So it might not be true. It might be that there’s some deep reason yet to be discovered why humans can never be immortal. Whereby immortal I mean, you know, that aging won’t kill us, but something else might kill us. If that comment in my book said that nothing is known that mandates that, then it’s not prophecy. But if I accidentally phrased it as a prophecy, then I’m wrong. Popperians shouldn’t be so embarrassed about being wrong as many people are.
Joe Walker
That’s very, very honorable of you. One of my favorite genres, David, is old books about the future. I like reading about how people in the past thought about the future. And I sort of collect these books mainly to remind myself not to prophesy. But some of the ones I have sitting on my bookshelf downstairs are Toward the Year 2018, which was published in 1968. There’s Lester Thurow’s book Head to Head. He was an MIT, I think political scientist, and it was a 1993 book that envisioned Japan and Europe as America’s great economic rivals in the 21st century, scarcely making mention of China. There’s Servan-Schreiber’s book The American Challenge, which envisioned American growth sort of continuing very aggressively into the 21st century. There is Kahn and Weiner’s book The Year 2000, which speculated on all of these future technologies that we would have. There is, of course, Ehrlich’s The Population Bomb. There’s Limits to Growth. There’s a book called The Coming War with Japan by Friedman and LeBard. But I’m curious, do you have any of your own examples of, favorite examples of failed predictions or doomsaying books that turned out to be false?
David Deutsch
Well, some of those I’ve actually read or at least seen some of those that you mentioned. Others are completely new to me. I haven’t heard of many of them. I was recently rereading 20,000 Leagues Under the Sea.
Joe Walker
I’m not familiar with it.
David Deutsch
Oh, Jules Verne, a science fiction book written about 1870, something like that. It’s amazing the things he gets utterly wrong and the things he gets amazingly correct. You know, electric light, submarines, and some of the things he gets very wrong. I saw there a nice example about Antarctica. It wasn’t known at the time whether Antarctica was land or frozen sea like the Arctic region. So in this book, they go in a submarine and they go and try and find the South Pole to see if it can be reached under the ice. He gives a wonderful argument for why there must be a continent there and not just ice. He says, near the South Pole, there are far more icebergs than there are near the North Pole. Icebergs can only form from glaciers which are on land. The glaciers in the Northern Hemisphere form in places like Greenland and so on, but not in the Arctic itself. That’s why there are fewer of them. Therefore, we must expect there to be an Antarctic landmass. That’s just so typical of explanatory reasoning. There’s no induction there. We found continents everywhere else, therefore there should be one in Antarctica as well. It’s an explanatory reasoning. It’s explaining the phenomenon of icebergs by something that absolutely isn’t icebergs. It’s a landmass in the middle of the Southern Ocean that’s never been discovered. That’s brilliant. I don’t know whether it’s even true. I haven’t looked it up. I don’t know whether there are more icebergs in the Southern Hemisphere. I don’t know. I don’t know. In the Southern Hemisphere, I guess it must be true by the same argument. My best guess is that it is true. You asked for examples of predictions or prophecies that turned out to be false. Well, I’m more impressed by the ones that turned out to be true. Oh, yes. He predicts that on the same trip, he predicts that in the future, hunting whales will be made illegal. This is 1870 or something like that. I think one thing that’s happened is that between the 19th and 20th century, speculative fiction, science fiction turned pessimistic. The 19th century was more optimistic, and when people speculated about the future, they’re more likely to have been wrong by overestimating progress than by underestimating it. In the 20th century, in the 20th century, there was a sort of congealing of the intellectual climate into a very rigid pessimism so that a prediction or prophecy could only be taken seriously if it was negative. There’s another book I thought you might mention. I don’t know exactly when it was written, but I remember it coming out called Will the Soviet Union Survive Until 1984? This was in the 70s. His answer was no, or rather it will survive until approximately 1984, then it will collapse. I remember this being vilified basically on the grounds that it was arrogant to assume that the West has it all right and that there’s nothing viable in the Soviet system and this is just arrogance on our part and so on.
David Deutsch
Whereas it’s not true. The book was just giving explanatory arguments about why this edifice could not survive. He was only five years out. It turned out to be correct, but he at the time was vilified.
Joe Walker
You mentioned that switch from optimistic to pessimistic visions of the future. You know, where you’ve got in the first half of the 20th century, like Isaac Asimov speculating about how wonderful the future could be and then by the second half of the 20th century, you’ve got movies like Terminator. You mentioned that that shift may have been caused by this congealing intellectual environment around pessimism. I’m not sure if I’m offering an alternative explanation or adding to what you’ve said, but what do you think of the idea that economic and productivity growth began to slow for whatever reason or reasons around say 1972, 73 and prior to that where we had this amazing period of growth, people were kind of extrapolating that into the future. So it made sense to be talking about flying cars only a few decades away because people had just gone from almost nothing to electricity, telephones, radios, flight. But when that growth started to slow, the pessimism kind of kicked in. Does that make sense to you?
David Deutsch
I think that happened, but I don’t think it’s… How can I put it? I don’t think there’s an inexorable evolution in this thing. I think it happened because of specific mistakes that got embedded in systems, particularly the academic world and governmental bureaucracy and from there into the wider society. So that, again, the idea that… How can I put it? That there’s something wrong with aspiring to make radical improvements, that this is hubris, or that this is dangerous, or that this will inevitably have side effects. This idea is very widespread. Various versions of this idea are very widespread and they have caused a slowdown in various areas. Of course, not in all areas. You’ve only got to look at computers to see that rapid improvement happened during that entire so-called period of stagnation. In many areas, improvement drastically declined. Declined not to zero, so we still have improvement all the time, but we don’t have rapid improvement. We don’t have rapid game changes happening anymore. I think that’s completely unnecessary and could be turned around if people change their attitude.
Joe Walker
That’s a very optimistic interpretation. I would like to believe that that is true, that there’s just something wrong in the culture. Mental problem, I guess, that we can tweak and get ourselves back on track. That would be much better than thinking that we’d somehow picked all the low-hanging fruits or something like that.
David Deutsch
Yeah, no. Right. That is the epitome of the wrong theory. I’m a physicist and so I can judge that in the context of physics. People have said, and I think the prevailing view is that the reason fundamental physics progress has slowed down is that we’ve picked all the low-hanging fruit. But that’s not true. There’s more low-hanging fruit than there ever was seen before. It’s just that picking it is stigmatized.
Joe Walker
That speculative fiction book from the 1870s you mentioned, do you remember the author’s explanation for why he thought whale hunting would be outlawed in the future?
David Deutsch
No. He says something like that whales have large brains and something like that. I forget. But he was not at all opposed to hunting sea creatures or land creatures. He’s quite okay with that there’s a scene where they encounter these other predators who are going to prey on the whales and they’re going to basically literally make mincemeat of them. He describes this in gleeful tones. It’s not that he’s against hunting, he’s against whale hunting in particular.
Joe Walker
Right, interesting. For you, a good explanation is an explanation that is hard to vary while still explaining what it purports to explain. But hard to vary by what standard?
David Deutsch
Ultimately, the standard is that conjectures that have been put forward or which are on the horizon for being put forward have been refuted. What’s more, they have been refuted in such a way that it’s not just that the particular theories have been refuted, it’s that their underlying assumption have been argued away. When I say refuted, I meant in this context argued away. That nothing like that could happen because if we were to find somehow a theory with that property that for example, that in reality the earth is flat and that it just looks round because light travels and it’s not in straight lines or something like that, then that would spoil all sorts of other explanations which those theories, the flat earth theories, do not address. And it looks as though they can’t address them. Now, just because it looks as though they can’t address them doesn’t mean they can’t. But we can’t switch to a theory that isn’t a good explanation because a theory that isn’t a good explanation is obviously false. It’s like, you know, I’m not going to step into the path of moving traffic on the motorway because it looks as though I’d be mashed by the next car that’s coming along. Now, it’s no good saying, well, you might be wrong. Yes, of course I might be wrong. It might all be a hologram and everything. But I can’t, it’s not rational to make decisions on the basis of what might be true. It’s rational to make decisions on the basis of what looks as though it’s true in the sense that the contrary theory looks false. Do you mind if I just make myself a cup of tea?
Joe Walker
Yeah, absolutely. I might get a water while you’re doing it.
David Deutsch
Yeah, because my voice is going. So unless I lubricate it. Right, I’m back. Great. Yes, you were saying?
Joe Walker
Yes. A few more questions about prediction. So in his book, The Precipice, the Australian philosopher Toby Ord takes a Bayesian approach to quantifying existential risks to humanity. He adds up the chance of various existential catastrophes befalling us in the next 100 years and reaches a rough overall estimate. The chance of an existential catastrophe befalling humanity in the next 100 years is one in six. He stresses that it’s not a precise estimate, but he thinks it’s the right order of magnitude. And the one in six estimate takes into account our responses to escalating risks. Question for you, David, should we use base rates like that to estimate the probability of existential risks and help prioritise which ones we address?
David Deutsch
Basically, absolutely not. We should not. But I have to qualify that by saying that in some cases, the probabilities can be known because they are the result of good explanations. So for example, we can calculate the probability that an asteroid from the asteroid belt will hit the earth in the next thousand years or something. Unfortunately, we don’t know the probability that an asteroid from somewhere else, from the Oort cloud or from somewhere outside the plane of the ecliptic or from elsewhere in the galaxy or from another galaxy. So we don’t know any of those probabilities. There’s no way of estimating them. So there is no way of using Bayesian reasoning to address them. I should also say another caveat is that because of the grip that Bayesian epistemology has on the intellectual world at the moment, people often phrase good arguments in Bayesian terms in order to give them the appearance of being strong arguments. Whereas in fact, they’re already strong arguments. They don’t need Bayesianism to justify them. So what you tend to get is a mixture of good arguments disguised as Bayesian epistemology with bad arguments that actually use Bayesian epistemology. Toby Ord’s book, I haven’t read it all, but it definitely makes this mistake of Bayesianism in both senses. That is, a lot of the book is good argument and good proposals, but some of it is just lost behind the mist of prophecy. Nick Bostrom’s vulnerable world hypothesis
Joe Walker
Is the hypothesis that there’s some level of technology at which civilization almost certainly gets destroyed unless quite extraordinary and historically unprecedented degrees of preventive policing or global governance are implemented. So in simple terms, the cost of destructive technologies fall and we have a diverse set of motivations in the population. There’ll always be a few crazy or malevolent people and it’s a near inevitability that those people use those destructive technologies to destroy civilization. Are you familiar with Bostrom’s vulnerable world hypothesis?
David Deutsch
Yes. I disagree with both the conclusion and the argument. Right. Though again, Bostrom is a wonderful writer and a lot of the things he says are very true. If you’ve read his letter from the future to the present, that’s the most uplifting thing I’ve ever read, I think, and highly optimistic. But I think the argument about technology and the dangers of technology is just wrong. So he has this analogy of the urn from which one takes white beads and black beads. Occasionally there are black beads and the white ones are, and these are technological discoveries. The white ones are ones that are beneficial and the black ones are the ones that destroy us. Sooner or later we’re going to hit a black one. And sooner or later we’re going to hit a black one. Unless we take drastic steps to make sure that first of all we take them out more rarely and second that we examine them very closely before actually deploying them. I think this is a recipe for totalitarianism, but even worse, it’s a recipe for, it’s precisely a recipe for civilization to be destroyed. Because a civilization is only going to be rescued by rapid growth of knowledge. Whether or not we take totalitarian draconian steps to try to rein it in. In terms of the analogy, the mistake is that every time we take out a white bead we reduce the number of black beads. So the probability calculation that’s implicit in that metaphor is a mistake. We become more resilient the more we know about, especially fundamental knowledge. Because fundamental knowledge can protect us from things that we don’t yet know about. Unlike specifically directed knowledge which has less of that tendency. Secondly, it’s not true that technology has made us more vulnerable. Because we are, our species, depending on how you count species, our species homo sapiens sapiens is, the terminology is changing too fast to keep track of it. But our species is one of six or eight or maybe more species that had the capacity to create explanatory knowledge. We know that because things like campfires require explanatory knowledge to form them. We know from the evolution of language, language must have been used before the structures in our throat adapted to make language evolved. It must have been the language use that made them evolve. It couldn’t possibly have happened the other way around. So we know that all these other species existed in the past and were capable of what we are, namely creating new explanatory knowledge. They’re all extinct except us. We know that our species almost went extinct at least once, but probably more than once in our past as well. So if we come nearer, every civilization before the civilization we called the West, now technological civilization, or every civilization before technological civilization was also destroyed.
David Deutsch
Some of them after 4,000 years, but I think that’s the longest a civilization has ever survived, depending on where you draw the line. But between its creation and what people regard as its destruction, destruction of the knowledge that kept it going, that’s nothing compared with the lifetime of a species. All those civilizations have been destroyed as well. All those species and all those civilizations could have been saved with just a little more knowledge from our perspective. A little more knowledge about hygiene to prevent plagues and farming and irrigation and that kind of thing to prevent being destroyed by climate change and so on. So a small amount of knowledge would have saved them. And on the other hand, not a single civilization was destroyed through creating too much scientific knowledge. So that’s never happened. So if you’re going to be Bayesian or if you’re going to pull these beads out of a hat, then even by that standard we should be pulling them out faster, not slower. Now, and as for the idea that a small number of people could destroy civilization, well, yes, but that’s not the right measure. We have a small number of people who could work on things that could destroy civilization, but we have a large number of people that could be working on countermeasures. Now, it could be argued, and I think there’s a very good argument for this, that we are not doing enough of that. That is, we’re not doing enough to counteract artificially caused pandemics, for example. Or as Carl Sagan put it, artificially caused meteor strikes. Now, and he said we shouldn’t be developing the technology for fending off asteroids or comets because it could also be used to destroy us. Now, I think that’s a mistake. We should be developing that technology and we should be developing it faster. And again, this is something that’s actually going in the right direction because, say, 20 years ago, the idea of an asteroid defense system was ridiculed. It was literally ridiculed and rejected just for being ridiculed. Whereas in that time, we have set up a rudimentary asteroid detection system which can detect asteroids and also there’s been research into how to fend them off when we do detect them. Now, they will not fend off asteroids coming from an unexpected direction outside the plane of the ecliptic, nor faster than we expect. We could be vulnerable to those just because we don’t have a fleet of nuclear-powered spaceships. And we’re going to be kicking ourselves if one of those heads towards us and we don’t have the nuclear-powered spaceships and it’s going to take us more than whatever it is a year to build them. Now, I don’t know what we could do if our lives depended on it. I guess we could do, like we did with the present pandemic, we could do things that were thought impossible previously. But there’s a level of things that we couldn’t do and we should be, if not making the fleet, we should be creating the knowledge to make the fleet of nuclear-powered spaceships and all sorts of other things. And as I said, the most important knowledge in this respect is fundamental knowledge. And fundamental knowledge is created by things like fundamental science.
David Deutsch
Fundamental science has been held back by the phenomenon we discussed before, that the academic world has been trapped in a sort of sargasso sea of bad assumptions, which have de-emphasized fundamental research in favor of incremental research. And the science funding system doesn’t work and the peer review system doesn’t work. And it all goes together to increase the number of scientists doing things that won’t save civilization and reduce the number working on things that will.
Joe Walker
I wanted to ask you this before, but what practical steps would you take to improve the incentives for fundamental research over incremental research?
David Deutsch
Well, I’m not an expert on science funding. And the reason for that is that, in part, I have tried to get away from the entire system, the entire academic system, from funding down to academic politics and so on. I don’t have an official position at any university and I’m not paid by anybody to do research. I write my books and I am an honorary member of various things in Oxford University, but not a paid member. So I don’t know how things are going. I only know the complaints that my colleagues raise. And they are all the same complaints that funding is highly bureaucratic at the moment. So if you have an idea for some research you want to do, you’ve got to submit it to a committee. The committee consists of 20 people, none of whom are experts in fields close to yours. Or even if they are, they’ve got a vested interest in not doing that kind of research but in directing it towards their kind of research, which is only natural. Now, it used to be, even when I was a student, it used to be that research funding was not done in that way. Research funding, I don’t know how the higher levels of it worked, but it was directed towards individual senior researchers and they disbursed the funds and chose their own graduate students and postdocs to do the research that they thought was important. And those of them who thought that fundamental research was important didn’t have particular projects that they wanted. They were looking for young people who had ideas to do fundamental research. That was certainly true of the bosses that I had when I was a student and when I was a postdoc. Now, that doesn’t seem to exist anymore. Now, it’s the scientific department that has its priorities and which tells the professors what to do. And the professors, we had some ridiculous situations a couple of years ago where we wanted to hire, well, one example was we wanted to hire a postdoc to work on foundations of constructor theory. And the reason we wanted to hire him is that he was the only person in the world who had proved a particular theorem and we wanted him to use his techniques anyway. It was impossible to hire him because we had to advertise his position and then make a case to the relevant researchers. And then make a case to the relevant committee. And what did they know about it? There wasn’t a box on the form. There wasn’t a box called constructor theory because constructor theory doesn’t exist yet. The whole point was that we’re trying to create this new field. And the reason it doesn’t exist yet is that it may not exist at all. It’s a fundamental conjecture. But it’s a fundamental conjecture that is thought worthwhile by me and several other senior people. That should be enough to fund something. And the same thing has happened with graduate students. And it’s ridiculous, by the way, that at least in the parts of the funding system that I see, if you’re a young person wanting to do research on a particular thing in a particular department, you’ve got to apply for the funding in one place, typically the government or some giant charity or something like that, and to get into the department in a different place. And so it can happen that, like in the example I gave, the department really wants you, but there’s no funding. So sometimes the senior people can arrange a weird arrangement where you’re funded for one thing, but you’re really going to do another thing. And I think in general, people who apply for grants nowadays are playing a game.
David Deutsch
You’re gaming the system. You’re trying to tick as many of the boxes as you can, and you’re trying to pretend that your research is directed towards those boxes, whereas actually it only satisfies the boxes incidentally, and it’s really directed towards something else that couldn’t get funding. Now, I shouldn’t go on and on about this because this is only a tiny facet of the overall problem. People like Michael Nielsen and Patrick Collison have investigated the problem at a deeper level, although they are inclined to… They have much more sympathy with the lower hanging fruit theory than it deserves, but at least they’ve gone into the problem in a broader sense than I have. So you should ask them.
Joe Walker
Yeah, there’s also a great book by Donald Braben called Scientific Freedom. Have you come across that?
David Deutsch
No.
Joe Walker
I recommend that to people as well. I think Stripe Press recently, Patrick Collison’s publishing house recently republished it. But yeah, I’m just getting frustrated listening to you. It just feels like we’re self-sabotaging as a species.
David Deutsch
Yes. Of course, the bad guys have no such restrictions.
Joe Walker
Right. There’s an asymmetry.
David Deutsch
Yeah. But really, the natural asymmetry is the other way around. The good guys have a natural advantage in this game, but not if we hogtie ourselves.
Joe Walker
Exactly. A couple more questions on prediction and probability. Yeah. Are you familiar with Phil Tetlock’s research on forecasting?
David Deutsch
No. And…
Joe Walker
No? Okay.
David Deutsch
Afraid not.
Joe Walker
I’ll skip over that then. Do you have any explanations as to why frequencies in certain situations can be approximated by probabilities?
David Deutsch
Yes. It’s because there is an underlying physical process for which, if we have good explanations of that process, we can use frequencies to predict probabilities, but not otherwise. Mm-hmm. And the usual case is otherwise. Right. That is, the usual case is that the frequencies are misleading, especially when something important depends on it.
Joe Walker
Why do you think it took people so long to come up with probability theory? Humans were gambling long before Cardano. The maths isn’t particularly difficult.
David Deutsch
Oh, I think the maths is quite difficult. I mean, if you’re going back to Cardano, it was only a couple of centuries before that, that Europe switched from Roman numerals and using letters for algebra. So, if you look at how people like Kepler expressed theories, it was amazingly cumbersome. But people like Kepler, people like Galileo were taken down by the system as it was then. So, as Bronowski says, after Galileo, scientific research in Southern Europe came to a dead stop, but it continued in Northern Europe. And then we had Leibniz and Newton and Descartes and so on. And they had their problems with authorities, but they were relatively free to pursue ideas, and they had fundamental ideas. And I don’t think it’s at all surprising that they didn’t invent, that probability theory wasn’t invented earlier. When probability theory was first invented by Cardano and Pascal and those people, it wasn’t misused. It wasn’t used in the same way as today. It was, I think, understood by all concerned that this was a theory of how to make a profit playing games, where the process of randomizing was always part of the explanation of why you should take the fact that three aces have already been dealt, that that changes the probability of the fourth ace. This was based on a physical understanding of the situation where a randomizing process had approximated probabilities. And nobody would have, I think, nobody would have tried to use this for predicting things like, you know, whether there’s going to be another continent in an uninhabited, unexplored part of the world. They wouldn’t have done that because they would only have been expecting probability to have this narrow range of uses. And it was only in the, I don’t know, again, I’m not a historian of ideas, but I think it was only roughly the middle of the 20th century when people like Jaynes and so on started advocating a much broader, no, no, it was before that. I think there were, there was already the beginnings of it in the 19th century. But anyway, a much broader interpretation of probability subjective interpretation of probability is, by the way, I haven’t mentioned this, these bad interpretations of probability are all subjective, including Bayesian epistemology. They’re all thinking that probability is not an attribute of the pack of cards, it’s an attribute of how we think of the, about the pack of cards. And that’s a terrible mistake which Popper attacks as well. I mean, he’s, he’s against subjective interpretations of anything except psychology itself.
Joe Walker
Speaking of subjective probabilities, and I know you are more interested in the ideas themselves than the history of ideas, but just as an aside, people often go back to Frank Ramsey when thinking about the birth of subjective probability, but it was recently when reading Popper that, God, it was in objective knowledge. It was, so it’s, for people who have the book, it’s page 79 of objective knowledge. It’s the second essay, Two Faces of Common Sense. But there’s a footnote of Popper’s, the theory is often ascribed to Frank Ramsey, but it can be found in Kant. And I got to be excited when I read that because I do get a bit sidetracked indulging in the history of ideas. I agree with you that it’s, I find it useful to the extent that it helps you understand the ideas themselves and kind of the debate between different ideas, even if only as like a memory device, I suppose. And yeah, I went back to Kant’s Critique of Pure Reason. And sure enough in there, he talks about using bets to quantify subjective probabilities.
David Deutsch
All the way back to Kant. Yeah. Yeah. It’s not surprising, I guess, because he was really into subjective interpretation of knowledge in general.
Joe Walker
Yeah. But I was like, wow, that’s a great pickup by Karl. Yeah. So the last question I wanted to ask you, David, was really something I just started thinking while we’ve been talking. And that is, I wonder whether you see the cultural malaise that’s been afflicting science and the kind of careerism and incrementalism as being at all a cause of the continued and perhaps increasing popularity of Bayesian epistemology? Because I guess with Bayesian epistemology, you can kind of keep like tinkering with existing theories rather than coming up with fundamentally new ones.
David Deutsch
So sociology is another kind of thing that I’m not particularly interested in. I don’t want to psychologize or second guess why people make the mistakes that they do. I would rather think that Bayesian epistemology is just one facet of a much larger thing. So it’s not that Bayesianism has caused all the trouble in the world. It’s that all the trouble in the world has caused Bayesian epistemology. However, it is striking that Bayesian epistemology, it’s all about increasing the authority of a theory, which in the big picture is all about increasing authority, which means that there’s, let’s follow the science as recently people have been saying about the pandemic and so on, as if science had some authority, had a moral authority or a finality or an indisputableness about it. And at the same time, it undervalues Bayesian epistemology, that is, undervalues criticism. So everything is focused in Bayesian epistemology in increasing our credence for something. And okay, we have a refutation that reduces it to zero. So it’s a kind of structureless conception of how theories can fail. According to that theory, they fail all at once and when they are refuted by experiment. Whereas in reality, in the Popperian conception, science consists entirely of criticism or rather of conjecture, which is a thing that we don’t know how to model, but theories don’t have a source other than conjecture. And the whole rich content of scientific reasoning comes in criticism, a small part of which is inventing experiments and doing them. But most criticism is structural criticism of the theory qua explanation. And most theories are rejected for being bad explanations rather than actually refuted. And even when there is an apparent refutation, we don’t take it seriously unless there’s an explanation for it. Again, the example I gave in my book is the fuss that was made when some people thought that they’d found neutrinos that travel faster than light. And they were thinking, oh, general relativity is refuted and so on. And actually, the explanation was that there was a faulty connector in some of their electronics. And that was it. That was the whole explanation for the neutrinos appearing to travel faster than light. Now, this is the Duhem-Quine critique of science in general. It does not apply to Popperian epistemology, but it does apply to Bayesianism. Because in Bayesianism, you never know whether your credence for the integrity of the experiment should be reduced or your credence for the theory should be reduced. Bayesian epistemology doesn’t give a criterion for which of those to choose. And nor does Popperian epistemology, but Popperian epistemology has an alternative account of what you should be doing, namely trying to find explanations. And then when you found the explanations, it’s not that your probability or your credence for them changes.
David Deutsch
It’s that their rivals become bad explanations.
Joe Walker
And that’s how we make progress. David Deutsch, thanks so much for your time.
David Deutsch
It’s been fun. Nice chatting.
Markdown