# 2011-03-10 - Oxford Transhumanists - How to Think About the Future [YouTube](https://www.youtube.com/watch?v=B53NX3sCdqg) Duration: 01:00:42 ## Transcript ### David Deutsch 00:00:00 - 00:06:13 Well, given the auspices under which this meeting is being held, I expect that most people in this room are in favour of progress. And more than that, I expect that the aspiration to rapid and unbounded progress does not evoke in most of you the nameless dread and even anger and sometimes willful irrationality that it does evoke in some circles. And therefore, I don't have to persuade you that, for instance, life is better than death. And I don't have to explain exactly why knowledge is a good thing and that the alleviation of suffering is good, and communication and travel and space exploration and ever faster computers and excellence in art and design are all good. Human flourishing is good. And also, contrary to the weight of conventional wisdom, it is not possible to have too much good. Okay, so, progress in all of these fields is related by the fact that it all depends on progress in science and in technology, but also for sustained progress over long periods, one also needs the rational decision-making institutions of an open society and also certain values such as tolerance and respect for the truth and rationality and optimism, which I will come to. I expect also that most of us here share that aspiration to progress in our individual lives and that we also share those values so that we expect the means of making progress to be reason and science, applied to solve problems and that it requires modes of thought, therefore, from which irrationality is completely excluded. So, I want to start with those assumptions and to explain what I think reason entails in regard to the unknowable future. The outgoing president of the Royal Society, Martin Rees, whom I have described as the finest all-round physicist of the day, wrote a book at the turn of the millennium called, some of you may have read it, it's called Our Final Century, meaning the 21st century. In America it was called Our Final Hour, with the subtitle, A Scientist's Warning How Terror, Error and Environmental Disaster Threaten Humankind's Future in This Century, On Earth and Beyond. So, Rees wrote in that book that the probability of civilization surviving the 21st century is about one half. He didn't argue for the exact number, but he wanted to stress that it was a lot less than one and therefore merits serious effects on policy, presumably with a view to increasing that probability. But I'm not sure whether he, he doesn't say whether he included in his estimate the probability that his book would actually achieve that effect by persuading people. And, see, there's already something paradoxical about the very concept of probabilistic prediction in regards to this problem. And that paradox is called the Cassandra Paradox. Anyway, his argument is that we are living at a unique moment in history, the moment when the range of technology is having effects and causing risks and dangers on the scale of the planet as a whole, and civilization as a whole, and our species as a whole. And so he lists lots of existential risks or some of them near existential, both from unintended consequences of progress, such as the escape of genetically engineered organisms from a laboratory which then go on to cause a pandemic of an incurable disease, but also malevolently intended side effects of progress, like terrorists using that kind of weapon or other weapons of mass destruction. ### David Deutsch 00:06:13 - 00:10:41 He's also worried about, and I think some of you are too, about intelligent robots, AIs, going rogue. And he says that nanotechnology is, could be even more threatening, I quote. And he had, that's just a selection of the risks he talks about. He points out that for this prophecy of doom to hold, that is prophecy of doom at the 50% confidence level to hold, it's not necessary for any one of these potential risks to be at all likely, because we need only be unlucky once. And we incur the risk afresh every time there is progress in a variety of fields. And Rees compares this with playing Russian roulette. Except that if you were right, it would actually be worse than Russian roulette, because with our accelerating rate of progress, we are pulling this metaphorical trigger more and more rapidly, more and more often. And we're also loading more and more bullets into previously empty metaphorical chambers. Okay, well, there are several mistakes there, which I don't have time to go into. But one mistake in particular is quite profound and very widespread, and is relevant to what I want to say today. Comparing the human condition with Russian roulette is a very false analogy, because the probability of winning or losing at Russian roulette is unaffected by anything that the player may think or do, provided that the rules are obeyed. It is a game of pure chance. But the future of civilization depends entirely on what we think and do. If civilization fails, there won't be something that just happens to us. It'll be the consequence of choices that people have made. And if civilization survives, then that will be because people have succeeded in solving the problems of survival. And that too will not have happened by chance. The reason that people confuse these two kinds of unpredictability is that, well, these two kinds of unknowability is that civilization and Russian roulette are both unpredictable, but for unrelated reasons. Russian roulette is just random. We can't predict the specific outcome, but we know what all the outcomes are and what their probabilities are, if the rules are obeyed. And therefore, if we were forced into some horrible choice between different kinds of games of Russian roulette with different numbers of bullets in chambers and so on, we could simply use game theory to work out what the best choice was. And we could do that perfectly mechanically, just as if the game were deterministic. The fact that it is random has no effect on how accurately we can compute what the right decision is. But in the case of civilization, the reason that trying to work out, trying to foretell the future of civilization is unpredictable, is that the knowledge that is going to determine not only the outcome, but all the steps towards the outcome and what outcomes are even possible, as well as what their probabilities are, that knowledge has yet to be created. ### David Deutsch 00:10:41 - 00:14:43 We don't know it, so we can't use it. And that would be true no matter how deterministic the process was. Even if we were in a simulation running on a deterministic computer and so on, that unknowability would still be perfectly true and the future would still be perfectly opaque to us in that respect. This makes all the difference. In game theory, which is also called decision theory, but I'm going to argue that it's not actually applicable to decisions of the type that I'm discussing, one assigns a probability and a utility to every possible outcome of every possible option that one has, and then one chooses the option with the highest expected value of the utility. But where one is ignorant of the possible outcomes and of all the possible intermediate steps leading to them, and all the options that will be available to the player at all those intermediate steps, then it's simply impossible to apply that decision theoretic method or model of making choices. And there is no fudge to get round this. The very use of the concept of probability in this context is invalid. In a sense, it's well known that the future is unknowable in this strong sense, but it's a fact that is almost universally underestimated. That's partly because it's like a blind spot. Just as our brains fill in the image of our eyes to disguise from us the fact that our eyes have blind spots, so it's hard to take the extent of our future blindness seriously. When we try to imagine the future of knowledge, our brains simply fill in an imaginary future which necessarily differs from the real thing in decisive ways. So just to bring home this point, I want to illustrate this with a thought experiment that's set in the past where we can use hindsight as a substitute for imagining the unimaginable. So suppose that someone in the year 1900 was trying to make plans for the 20th century and trying to use the best knowledge existing then to predict the factors that would either threaten civilization or be necessary for the further progress of civilization. They couldn't possibly have guessed that some of the biggest threats and also some of the biggest opportunities would come from the properties of the element uranium and from its transmutation into plutonium and into other elements and from the heavy isotopes of hydrogen. They couldn't have guessed any of that because radioactivity had only just been discovered, almost nothing was known about it. Transmutation was still a standard example of a superstition that alchemists used to believe in which science of chemistry has now discovered to be impossible. Even though Rutherford and Soddy actually demonstrated it in 1901, but we're talking about 1900. And isotopes weren't discovered until 1913 and plutonium wasn't discovered until 1940. And it's not just nuclear physics. ### David Deutsch 00:14:43 - 00:19:05 Other entire fields such as aeronautics and computer science and biotechnology shaped the 20th century. And just as no one in 1900 could have foretold the content of those fields, in many cases because they couldn't have foretold the existence of those fields, so our own future will be shaped much more than that by knowledge that we do not yet have. We can't predict even the problems that we'll encounter, let alone the solutions and the attempted solutions and the failures and the consequences of all that happening. People in 1900 didn't consider nuclear power or the internet unlikely, they just didn't conceive of them at all. Okay, well that was a thought experiment, but more or less the actual experiment was done in 1894. Albert Michelson, a Nobel Prize winning physicist at the Michelson-Morley experiment, attempted a prophecy that required the same kind of foreknowledge as the one that Rees attempted, except that Michelson was trying it for a much simpler sub-problem, that of prophesying just the future of theoretical physics. And his conclusion was as follows, I will read it. The most important fundamental laws and facts of physical science have all been discovered. And these are now so firmly established that the possibility of their ever being supplanted in consequence of new discoveries is exceedingly remote. Our future discoveries must be looked for in the sixth place of decimals. Okay, well, spectacularly wrong, right? But we have to think, Michelson wasn't stupid, you know, what exactly was he doing when he judged that there was only a quote, exceedingly remote chance, probability that the foundations of physics as he knew them would ever be superseded? Well, he was drawing conclusions from the best available knowledge. But that best available knowledge was none other than the physics of 1894. It just isn't the case that he worked out that relativity probably wouldn't be invented 11 years later. And that quantum theory was probably false. And that the universe probably wasn't expanding and the Big Bang probably didn't happen. And so on for almost the whole of what we now consider fundamental physics and chemistry. He just didn't conceive of any of those possible outcomes at all. And a century earlier, Lagrange, mathematician and physicist, had made exactly the same mistake. He remarked that Isaac Newton was not only the greatest genius that ever lived, but also the luckiest. Because, quote, the system of the world can be discovered only once. Lagrange's own work helped to pave the way for the replacement of Newton's system of the world. And so did Michelson's. Isn't that ironic? Okay, the reason for all this is that no explanation, however powerful, can possibly predict the content of its own successors. And therefore, nor can science predict any phenomenon whose course is going to be affected by the growth of knowledge, by the creation of new ideas. ### David Deutsch 00:19:05 - 00:23:34 Not just knowledge, but also by the state. This is a fundamental limitation on the reach of scientific explanation and prediction. And when planning for the future, it is vital to come to terms with this. So, okay, from now on, like I'm going to follow Karl Popper, and I'll use the term prediction for conclusions about future events that follow from good explanations, and I'll use the term prophecy for anything that purports to know what is not yet knowable. Trying to know the unknowable leads inevitably to error and self-deception. In particular, a prophecy always has one of two structures. It either consists of whistling in the dark, saying something like, you're going to meet a tall, dark stranger and live happily ever after. Or, and this is especially true if it's a serious attempt to extract unknowable answers from existing knowledge, then it's going to be biased towards bad outcomes. And the basic reason for that is that, as I said, the growth of knowledge is good. And so that kind of prophecy, which can't imagine it, is going to be biased against prophesying good. A famous example is that in 1798, Thomas Malthus argued that the 19th century was going to see a permanent end to human progress. He calculated that the exponentially growing population at the time, which is a consequence of various kinds of progress, was about to exceed the planet's capacity for producing food. And that this was not an accidental misfortune. He believed that he discovered a law of nature. Because, as he argued, on the one hand, the net increase in population in every generation is proportional to the existing population, and therefore population increases exponentially. But on the other hand, when food production increases, for instance when previously uncultivated land is brought into cultivation, then that increase in food production is not proportional to the existing population. And so it must be overtaken by the exponential increase. And it must be overtaken sooner rather than later. So Malthus concluded that he was living at a uniquely dangerous moment in history. Remember that. That's an idea that's been quite a common feature of pessimistic philosophies over the centuries. In the end, in the event, throughout the 19th century, the population exploded just as Malthus had predicted. Yet the end to progress that he had prophesied did not happen. Quite the contrary. And that was in part because food production increased even faster than the population. And then during the 20th century, both those increases continued even more with food production again winning. Now do you see what happened there? Malthus quite accurately foretold one phenomenon, the population explosion, but he entirely missed the other phenomenon, the explosion in food production. Why? Because in 1798, the forthcoming increase in population was much more predictable than the even larger increase in food supply. ### David Deutsch 00:23:34 - 00:28:08 Not because it was in any sense more probable, but simply because it depended less on the creation of knowledge. By ignoring that structural difference between these two phenomena that he was trying to compare, he slipped from educated guesswork into blind prophecy. He thought that he was making predictions from the best knowledge available. But in reality, he was just allowing himself to be misled by the ineluctable and permanent fact of the human condition, that we do not know that which we have not yet discovered. So, given that some of the determinants of future events, and more and more of them, are going to be beyond the reach of scientific prediction, what is the rational approach to the unknowable? Well, we have to start by unwinding all those false assumptions about estimating probabilities and playing games and knowing the unknowable. And we have to return to the basics with which I began here. Reason and science are the means to progress, but they are not means to prophecy. I must just point out one more thing to set the context. The Russian roulette argument would be valid if there were no progress, or if progress were very slow. Because, well, there are several reasons. One is that existing technology contains dangers as well. Civilisations have been destroyed before with primitive technology. And merely pulling the trigger less often doesn't change the inevitability of doom. And not all potential catastrophes are side effects of progress at all. For a given catastrophe, just imagine something like a pandemic or an imminent comet strike, but unforeseen, how much harm it does depends on how rapidly knowledge can be brought to bear while it's happening and afterwards. Therefore, one of the most important uses of technology is to counteract disasters and to recover from disasters, both from foreseen and unforeseen evils. Therefore, the speed of progress is one of the things that is a defence against catastrophe. And note also that in regard to intentionally caused evils, it is perhaps the main defence. The speed of progress is one of the things that gives the good guys the edge over the bad guys. Because good guys make faster progress. And the more progress slows down, the more level the playing field becomes, and the more the bad guys can choose their weapons and their moment to do harm. So, rapid continuing progress is indeed desirable. And so the question is only, is it possible? So I'll begin with the most fundamental kind of knowledge, universal laws, both scientific and philosophical. Now, that kind of knowledge consists of explanation, assertions about reality and how reality works. Is that kind of knowledge bounded? And I suppose we have to ask first, are there these universal laws? And if there are, are they knowable? Is the world explicable? ### David Deutsch 00:28:08 - 00:32:07 Because again, if there's a bound on the explicability of nature, on the amount of knowledge that we discover, for whatever reason, I mean significant knowledge that affects what we want to plan for, then prophecy is only impossible until we hit that bound. After that, it would be Russian roulette from then on. So what sort of, if such a bound did exist, what sort of bound could it be? It can't be caused by sheer hardware limitations of the human brain. And that's because of the universality of computation, which ensures that hardware limitations can only be essentially a deficiency of speed or memory, and not limitations on content. So in that case, we could still understand all relevant explanations if necessary with the help of computers, or with the help of RAM expansions for the brain. There's nothing fundamentally new there. We've been understanding the world for centuries with the help of pencil and paper, which is logically the same thing. As Einstein remarked, my pencil and I are more clever than I. Well, another possibility that some people claim is that we may be qualitatively unable to understand some explanations, or that the world just isn't explicable beyond a certain limit. J. B. S. Haldane famously expressed this as, the universe is not only queerer than we suppose, but queerer than we can suppose. Rather dramatic. And Richard Dawkins even has an evolutionary argument for that. He said, his argument goes like this, in our ancestral environment, being able to detect things like the approach of predators improved our ancestors' chance of surviving to have offspring. But evolution did not waste any resources on giving us the ability to detect phenomena that were not relevant to our survival. Which is why, that's exactly why, we can't, for instance, see quasars with the naked eye, even though in the cosmic scheme of things quasars are very important. So, Dawkins argues, there's no reason to expect our brains to be any different from our eyes in this respect. They too evolved to cope with, to understand phenomena on human scales of size, time, energy, and also just parochial factors. But most phenomena in the universe happen far above or below or outside those scales. So, just as our senses can't detect quasars, or neutrinos, or most significant phenomena in the cosmic scheme of things, there's no reason to expect our brains to understand those things. And it follows then that to the extent that our brains already do understand some of those things, we must have been lucky. So the theory goes. And a run of luck can't be expected to last for long. So scientific progress can't be expected to exceed a certain limit defined by the biology of our brains. And we must expect to reach that limit sooner rather than later. That's always the case when these exponential growth things are being compared. ### David Deutsch 00:32:07 - 00:37:10 Beyond that limit, the world stops making sense, or seems to, from our point of view, be the same catastrophe. But if Haldane were really right that we live inside a little bubble of explicability in a great inexplicable universe, then the inside couldn't be really explicable either because the outside is needed in our explanation of the inside. In fact, Haldane's own argument is an example of that, of why we can allegedly not understand certain things. So it implies that even our parochial experience would only seem explicable if we carefully refrain from asking certain questions. Now doesn't that bear an uncanny resemblance to the pre-scientific intellectual landscape with its distinction between the earth that was built for human convenience and a heaven that's beyond human comprehension? So this is trying to take us back to an archaic, anthropocentric world view. And in fact, any assumption that the world is inexplicable just leads to extremely bad explanations of the easily variable, just-so-story kind that characterizes myths. Because by definition, if we're in a finite bubble of explicability, then no hypothesis about the outside can be a better explanation than just the claim that Zeus rules there. And therefore, also, that he rules here. And so Haldane's theory is nothing other than an appeal to the supernatural. If we wanted to incorporate into our world view an imaginary realm of which we have no evidence and can have no evidence, but which nevertheless affects us, then we need never have bothered to abandon the myths of antiquity. And we wouldn't have. Okay, so the world is explicable. So far I've been discussing pure understanding, but that mostly affects the world via technology. Pure understanding, we've just established, is possible. But why is technological progress possible? In principle, we could understand everything and understand why we're going to be wiped out, but not be able to do anything. What is the necessary connection between fundamental understanding and technological progress? Well, consider the set of all conceivable transformations of physical objects. I mean conceivable, not necessarily possible. I mainly have in mind transformations for the better, but consider the set of all conceivable transformations. Now, some of those are forbidden by the laws of physics, like faster than light communication. And some, like the formation of stars out of hydrogen, happen spontaneously. And some, and actually this is the vast majority of transformations, such as converting air and water into trees, or converting minerals into radio telescopes, are possible, but they happen only when the requisite knowledge is embodied in suitable physical objects, such as genes or people. And these are the only possibilities. That's to say, every conceivable physical transformation is either impossible because of the laws of physics, or achievable given the right knowledge. That there isn't a third possibility. And that is a momentous dichotomy, which is entailed entirely by the scientific worldview, because if there were transformations that technology could never achieve, no matter how much knowledge was brought to bear, then this would itself be a testable regularity in nature. ### David Deutsch 00:37:10 - 00:41:57 You could predict that whenever that transformation was attempted, it would fail. So that would be a law of physics, which is a contradiction. It follows that everything that isn't forbidden by laws of physics is achievable given the right knowledge. There are no insuperable obstacles to transforming nature other than the laws of physics. And that is almost what I have defined there as the principle of optimism, namely all evils are due to lack of knowledge. The only remaining gap there is the question of whether any of the prohibitions imposed by the laws of physics are inherently evils. For example, some people imagine that death is somehow required by the second law of thermodynamics. Oh dear. Well, a non-silly example is the argument that the finiteness of the speed of light must eventually put an end to exponential growth, because the volume of the sphere that we occupy can't possibly increase faster than the cube of the time taken. But that is actually misleading in several ways. One simple way is that because of time dilation, that speed of light barrier is never experienced as a barrier by the fast-moving observer. You could cross the galaxy in a day, subjectively, if you moved fast enough. But more importantly, from the point of view of what I'm discussing, that argument misconstrues the nature of exponential growth in regard to knowledge. Basically, knowledge is a replicator. It's an abstract replicator. Not only does it reproduce itself, it undergoes variation, recombination, and evolution. And so, the amount of knowledge in an environment of rational thought that allows it to grow, grows exponentially. But that's exponentially relative to the speed of computation. That's what counts. Not the speed relative to proper time or cosmological time. It's improvement per computational step, or per unit thought. And that's what's experienced subjectively by the creators of knowledge and the users of knowledge. With suitable technology, we could always adjust that down to any convenient value that was necessary to make it subjectively still exponential growth. And in the cosmological long run, it depends on what cosmological model is true, in the long run that might be what we do. And by the way, it's not directly relevant here, but I thought to this audience I'd better mention as well, that for the converse reason, it's a mistake to think of the so-called singularity as being a shock, where we suddenly find that we can't cope with life because iPhone updates are coming every day, or every hour, or every second. And that's a mistake, because when progress reaches that speed, our technologically enhanced speed of thinking will have increased in proportion. And so subjectively again, we will experience mere exponential growth, in the sense that I just defined, which is pleasant, satisfying, and never too much of a good thing. In fact, no evils are known to be built into the laws of physics, and we shouldn't expect any to be discovered, because the way the growth of knowledge works is that basically growing creativity is pitted against fixed obstacles of nature. And that's basically why creativity always wins eventually, or can always win eventually, if applied by optimists. ### David Deutsch 00:41:57 - 00:47:07 If there were an obstacle that could forever evade creative attack, that would mean, because of the connection between technology and fundamental understanding, that there was also a bound on fundamental understanding. And as I said, the belief that such a bound will exist without any evidence is rank superstition. So, given optimism now, how do we plan, how do we formulate policies for the unknown? And the basic answer is analogous to what is already standard practice within science, which Karl Popper described very well as follows. I'm going to read an abridged quote. This question has always been asked in the spirit of what are the best sources of knowledge, the most reliable ones, the ones which will not lead us into error. I propose to assume instead that no such ideal sources exist, and that all sources are liable to lead us into error at times. And I propose to replace, therefore, the question of the sources of our knowledge by the entirely different question, how can we hope to detect and eliminate error? In other words, he's saying, assume that errors will happen and try to increase our ability to detect it once it has happened, after it has happened. He applied the same insight to his political philosophy, with his realization that the fundamental traditional question of political philosophy, namely who should rule, begs for authoritarian and therefore progress impeding answers. And that the rational question is, how can we set up institutions in such a way that bad rulers and bad policies can be removed without violence, and indeed as easily as possible? So applying that idea, that basic idea to our question, well those decision theoretic strategies that I've argued can't work, are the same as those non-existent sources of reliable knowledge that Popper was referring to. And those non-existent reliable ways of choosing good rulers and good political policies. Consequently, optimism demands that we not try to extract prophecies of everything that could go wrong in order to forestall it from our existing scanty and misconception laden existing knowledge. Instead, we need policies and institutions that are capable of correcting mistakes and recovering from disasters when they happen. When, not if. So they must, among other things, facilitate rapid knowledge creation, including technological knowledge, but also including wealth, defined not as bank balances, but as just the repertoire of physical transformations that we are capable of causing. In addition to those general aspects of policy, optimism also has specific implications, I think, for every plan for the unknowable. For society, for individuals. For instance, it tells us that the way to prevent that nightmare of the rogue AI apocalypse is not to try to enslave our AIs, because if the AIs are creating new knowledge, and that's the definition of an AI, then successfully enslaving them would require foretelling, prophesying, the ideas that they could have, and indeed the consequences of those ideas, which is impossible. So instead, just as for our fellow humans, and for the same reason, we must allow AIs to integrate into the institutions of our open society. And for similar reasons, optimism tells us not to be gullible about genetic, sociobiological explanations of human choices, even in the past and present, but certainly not for the future. ### David Deutsch 00:47:07 - 00:50:29 Because just as human choices can't be predicted when they involve creating new knowledge, so they can't possibly be encoded in our DNA, pre-encoded. Okay, so another issue. Optimism tells us not to hide from the extraterrestrials. The idea that there could be beings out there that are to us as we are to animals is, again, quite straightforwardly, a belief in the supernatural. It is to forget that there is only one way of making progress, namely reason and science. We've agreed on that. And the only moral values that permit sustained progress are the objective values of an open society and, more broadly, of the Enlightenment. No doubt the ET's morality would not be the same as ours, but nor will it be the same as that of the 16th century conquistadors. It will be better than ours. So all we have to do is want improvement and know, because of optimism, that it is achieved by knowledge so that any culture shock or other evils, which no doubt will result from any contact with ET's, can be overcome with even more knowledge. Over the centuries, the term optimism has had a wide variety of meanings. It was originally a philosophical term, like that, but originally referring to that notorious theory of Leibniz's that we live in the best of all possible worlds, and that all apparent counter-examples to that are actually necessary for some greater good, which was therefore an incomprehensible greater good. And therefore that theory is, in my terminology, almost pure pessimism. Since then, the word optimism has come to mean all sorts of other non-philosophical things, such as having a sunny disposition, which is certainly a lot better than Leibniz's theory. But optimism in this sense that I have argued for is not a feeling. It's not a bias or spin that we put on facts, like half full instead of half empty. Nor on predictions, it's not hope for the best, nor blind expectation of the best. In some senses quite the contrary, we expect errors. It is a cold, hard, far-reaching implication of rejecting irrationality, nothing else. Thank you for listening. ### Oxford Transhumanists host 00:50:29 - 00:50:47 I'm sure there will be a couple of questions after that. Yes. ### Audience questioner 00:50:47 - 00:51:35 Hi David. Yes, I had two questions, one is a sensible one, and one is maybe a little bit facetious. So you were saying that there's no reason we should hide from ET, and I guess you're alluding to perhaps Stephen Hawking or people who suggested that we're all doomed because we're just going to be annihilated by some far superior race. I like your answer that in order to have become superior they have to face these kinds of... But what about the Borg? The Borg, for those of you who aren't Star Trek nuts like me, are a bit like a disease. There are a lot of advanced societies out there, but they are something that's gone wrong and that can exist and grow at the expense of things like a cancer. So shouldn't we be worried about them at least? ### David Deutsch 00:51:35 - 00:51:49 Okay, so I'll start with my facetious answer, and then the serious answer. The facetious answer is the Borg are fictional. ### Audience questioner 00:51:49 - 00:51:52 How do you know that? ### David Deutsch 00:51:52 - 00:52:56 To be more serious. The Borg way of life is a way of life. It doesn't create any knowledge, it just continues to exist by assimilating existing knowledge. That's actually very like what the Roman Empire did. It respected knowledge, but it didn't create knowledge. So long as it was expanding and conquering other cultures and taking the best from them, it could survive. When it stopped, it began to decay and went away. So the Borg way of life is actually, although it doesn't look like it, like the Roman Empire didn't look too vulnerable in the first few hundred years, but the Borg way of life is actually a fixed way of life. And it's never going to win in the long run against an exponentially improving way of life. Actually I think that is even shown in the Star Trek series. Any more questions? Do you say you have a second question? ### Audience questioner 00:52:56 - 00:53:28 No, no, no. I'm sure there are other questions. David. Wonderful talk. I was just one particular question. You said that evils aren't built into the laws of nature, but I just wonder how you can reconcile the principle of optimism with your post-Everett quantum mechanics, because presumably in the vast majority of Everett branches that support life, then intelligent beings aren't going to arise that are capable of getting rid of that. ### David Deutsch 00:53:28 - 00:54:13 Well, yes, you might as well talk about the past or the present. We haven't cured cancer yet. In a sense, cancer is an evil built into the laws of physics, because we couldn't have evolved without some errors in our design, which we eventually could have corrected. That doesn't contradict this. That evil is due to lack of knowledge. Now there are some universes in which there's a drastic lack of knowledge because of bad luck or whatever, and in those there are more evils than in our universe. There are some universes in which there's more knowledge and they have fewer evils. But this is a truth about the multiverse as a whole. It's a law of nature. ### Audience questioner 00:54:13 - 00:54:19 If all evils are due to lack of knowledge, does that mean in the future insurance won't be necessary? ### David Deutsch 00:54:19 - 00:55:09 No, because the kind of converse of that is that there will always be errors, and there's no upper limit to the harm that can be done by an error. This version of optimism is not so much foretelling anything. It's a way of understanding error. It's saying that error isn't insuperable. It's not something that has to be that way. It has to be that way, rather. It's something that knowledge can overcome. So yes, there will always be insurance consequences. ### Audience questioner 00:55:09 - 00:55:18 I guess once we all have lots of backups, then insurance won't be that important. ### David Deutsch 00:55:18 - 00:55:20 You could insure the backups. ### Audience questioner 00:55:20 - 00:55:39 Yeah. Once you have, say, 10 to the 20th. You also seem to me to imply that the likelihood of error stays constant over time. Is that right? ### David Deutsch 00:55:39 - 00:56:29 I don't think so. Well, first of all, I certainly wouldn't put it in terms of likelihood, because likelihood is illegitimate when discussing the future of knowledge. But I think not, because I think, as I just said, we're fallible, and not only are mistakes always going to be made, but there's no upper bound on the size of mistakes that are going to be made, any more than there's an upper bound on the amount of knowledge that can be created. There are no infallible sources of knowledge. If we knew that there's an upper bound on the number of mistakes that could be made, then once we had reached that limit, like in any particular year, we'd know that we'd be infallible for the rest of the year. So there can't be such an upper bound on the size of mistakes. So we could be destroyed altogether and still be compatible with that. ### Audience questioner 00:56:29 - 00:56:57 I have one question. It seems like we still need to make some kind of forecast in a quantitative way about the future in order to decide how to act. You warn against making probability estimates about knowledge that we don't have. But supposing you were in Malthus's position, say with the contents of the talk, but otherwise only Malthus's knowledge, and you have to make some kind of recommendation to government or some kind of decision, how would you act? ### David Deutsch 00:56:57 - 00:58:34 Okay, well there are two things. Not everything is affected by this blindness. It's only the things that are going to be affected by knowledge. So we can, on the assumption, let's say, that somebody isn't going to work out a way of switching off the sun and isn't going to use that method in the next year, we can use ordinary astronomy to predict what the sun will do. And similarly, and some things are half and half, like I said, the growth of population doesn't require all that much knowledge because the knowledge to create more people is going to remain the same throughout the 19th century. And so that could be, we can always make mistakes even with scientific prediction. But what I would advise then and now is that policy should be to create knowledge that is broad and deep so that it has a chance of coping with unknown unknowns and sometimes unknowable challenges. So go for broad and deep knowledge. And before that, at the beginning of the 19th century, there was just the basic thing of improve the open society structures so that we can indeed correct errors and that there aren't aristocrats who can arbitrarily do stuff. ### Audience questioner 00:58:34 - 00:59:24 Thank you. Yes. I like the idea very much. What worries me is that fear, which is the opposite of optimism, stops people thinking, and we can get metastable states of opinion and so on, like we have on the subject of nuclear radiation and so on, where people think they understand and they don't. And the lack of knowledge is very serious and is going to affect us very badly. What we need is knowledge and optimism at the time that you... And so there are quite a lot of things where we need to unwind and re-educate ourselves because we've been making mistakes in the past. ### David Deutsch 00:59:24 - 00:59:37 I couldn't agree more. And one kind of knowledge that we need is knowledge of how to prevent this... you had a nice phrase for it. ### Audience questioner 00:59:37 - 00:59:39 Metastable. ### David Deutsch 00:59:39 - 00:59:53 Yes. A bubble of fear. I mean, I often write at the outset that fear is everywhere. Yes, so I agree. ### Audience questioner 00:59:53 - 00:59:58 Do you think there is a limit to the knowledge that the brain can understand? The human brain, I mean? ### David Deutsch 00:59:58 - 01:00:31 Well, obviously the brain has a finite information-carrying capacity, but that can be enhanced by technology. And as I mentioned, we already are enhancing it by technology routinely all the time. And I also argued that there can't be a fundamental limit to what we can understand because of the combination of the universality of computation and the argument that... Assuming that there are incomprehensible things is tantamount to a belief in the supernatural. So, in short, no. ### Oxford Transhumanists host 01:00:40 - 01:00:42 Thank you.