Roko's Basilisk: The Most Terrifying Thought Experiment

What I'm about to tell you is considered by some very serious thinkers to be an information hazard. Information that just by knowing it is potentially harmful to yourself and to others. Make no mistake, I am making an ethical decision just telling you about this idea. I feel okay doing so because I don't take the risk seriously. However, the disclaimer is there. If you wish to continue, keep watching because something wants to see you. Now entering the facility. We begin with something of an internet legend. You see, a few years ago on the rationality and philosophy blog LessWrong, a user named Rocco or Rocco, I'm going to call it Rocco because that's just sounding cooler to me, posted a thought experiment. Upon dissemination of this thought experiment to the rest of the website, the founder of the website, Eliezer Yudkowsky, called the post stupid. He then called Rocco an idiot, deleted the post, scrubbed it from the website and prevented any further conversation about this thought experiment. Rocco had come up with a thought experiment in the form of a basilisk named after the mythical beast who could apparently kill some intrepid hero with its gaze alone. And Yudkowsky deleted the post because of its potential to gaze at a user on the website, metaphorically speaking, and do harm to them. He decided to slay, not a scaly beast, but an idea. And some users of the LessWrong website did report great mental distress and even nightmares after considering this thought experiment. Again, I'm deciding to tell you about it because I do not take it seriously and I don't think you will either, but if you do not handle existential dread to the nth degree very well, I suggest you take the hyper rail back out of the facility. The thought experiment that is Rocco's basilisk starts like this. Suppose that in the future we are able to create a hyper intelligent AI, something straight out of the singularity. We then ask that AI, as we might, to help us optimize all aspects of human civilization. But then for reasons unknowable to beings like us compared to its intelligence, it decides that the first step towards optimization starts with inflicting eternal torment on every single human being that didn't want it to come to fruition or didn't help it come into existence in the first place. After all, how can you optimize without the optimizer? How would the basilisk know that you did or didn't help it come to pass? Well, as we learned in a previous episode of this program, a sufficiently advanced artificial intelligence could literally simulate all of human history, every single human ever, faster than that. So this basilisk might be able to meaningfully recreate you every single thought you ever had or will have and then be confident in its prediction of what you did or didn't do in the past to judge you in the future. Here is where the basilisk really rears its head. If its creation and its revenge are a real possibility in the future, shouldn't you, all of you, right now start advocating for its creation or help build it yourself to escape eternal torment and anguish? Well, the good news is that if you've never had this thought, the dilemma never presents itself to you and you can escape it. The bad news is that because of me, now, you can't escape it. The basilisk has seen you. Now, what are you going to do? Information hazards, as defined by philosopher Nick Bostrom, the guy who came up with simulation theory, broadly fall into two main categories. The first is information that's potentially hazardous to others. The second is information that's potentially hazardous to yourself just by knowing it. The first category is pretty easy to think of. If someone were to give someone else the information on how to build a nuclear bomb in their basement, that would probably be catastrophic for a large number of other people. The second category is a little harder to think of, but I'm sure you've had experience with it. Take a health risk. If you had your DNA sequenced by some service, for example, maybe it told you that you had an elevated risk of cancer and you didn't really want to know that and now you're stressing out about it. Or more theoretically, think of some omnipotent being coming to you in the middle of the night and telling you the exact day and way that you are going to die. I'm guessing having that information would change your entire life and be very stressful. Or think about, I don't know, some guy on the internet with a man bun putting you in the sights of a malevolent AI who's going to torture you forever just by thinking about it once. That sounds like it could be stressful too, I guess. Roko's Basilisk is even more sinister than it seems at first glance, and we can demonstrate that with another thought experiment. Follow me here, called Newcome's Paradox. So say I come to you with two boxes labeled A and B. Box A is transparent and inside you can see a thousand dollars. Ten Benji Boys. A stack, as the kids say. What's in box B is not apparent to you, because box B is not transparent, it is opaque. And I tell you, what's inside depends on you. You see, I'm an amazing predictor, and I have the ability to tell the future. And if I predicted you will choose box B, there's a million dollars in box B for you. Ten thousand Benji Boys. Mucho escalote, I think the kids say. But if I predicted you choose box A and B, then box B is empty and you only get that one sick stack of paper, or cheddar, or whatever they're calling it these days. Now you can only choose box A and B, or box B. So what do you do? What should you do? Well, I don't expect an answer in the form of an essay from you, because game theorists and philosophers have been debating this paradox for years. But what you choose ultimately depends on how good of a predictor you think I am, right? Well, enter the Basilisk. Say that the Basilisk presents you with a similar choice. It wants you to take box A and box B, though, together, because that, metaphorically speaking, is choosing to help it come into existence. It then tells you that if you only take box B, the other option, you will experience eternal torment. Like, oh, I don't know. It will insert wires into your brain and pump in electrical signals that equate to living a billion lifetimes with every nerve in your body screaming in the absolute zenith of possible pain. I don't know. Something like that. Now, if you were convinced that the Basilisk was a perfect predictor, which it could be, you would most likely take both boxes. You would do what it wants. Because Roko's Basilisk knew that you would take both box A and box B when threatened, box B never had to actually contain any torment at all. You've now experienced future blackmail. If you think there's any real possibility that Roko's Basilisk could slither its way into your possible future, then logically, if you don't want to incur its wrath, you should start bringing it into existence right now. And notice that you're now being controlled by a future entity that doesn't exist yet simply because you had a single thought about it. You will have acted in its past the way it will have wanted you to in its future. Yeah. Let that break your brain for a second. What are you doing? I'm playing with a voice synthesizer. Why? It's almost ready. Right. Hope I know what I'm doing. It's hard to imagine that anyone would really think a Basilisk could exist. A real creature that could kill you with a single gaze. That is, unless you were a believer in the ancient theory of extramission. This idea, developed by very smart people like Plato and Ptolemy, said that our eyes don't see stuff by interpreting electrical signals from photons hitting the backs of our eyes. No, we see stuff by sending out eye beams that go and touch stuff and then tell us what they touched. They didn't sound like that, but like actual eye beams. It's this idea that led to Sir Thomas Brown in the 1600s musing on how a Basilisk might actually work. He said, it is possible that the visible rays communicate the subtlest of the poison and to a beast, a man infecteth their brain and then is communicated to the heart. He didn't sound like that either. Is the idea correct? No. Is it fun to talk about mythical beasts and weird accents? Yeah. I apologize if you're kind of freaking out right now, but the Basilisk gets even worse. Not only does thinking about it implicate you in its future wrath, thinking about it makes it more likely to exist in the future in the first place. You see, the more people that think about this thought experiment, the more people might choose to start bringing it into existence, and now it's more likely to exist in the future. Even if it was never going to exist, thinking about it right now as we're doing increases the chances that it will. This is why this is often deemed the most terrifying thought experiment, a kind of idea virus. And if you take this idea seriously, you can see why Yudkowsky and LessWrong handled this info hazard in the way that they did. I do not take it seriously. I do not think we will ever reach this kind of singularity, much less create AI with these kinds of capabilities or give it an optimization protocol that's similar to this one. And if we all just agree right now that we're not going to bring it into existence, then nothing will happen. Besides, I've already built one of my own. The Basilisk has its eyes on you now. What are you going to do? Until next time. Thank you so much to the very nerdy staff at the facility. How are you all doing? You're not freaking out too much, right? For the direct and substantial support in creating this video. Today especially, I want to recognize research assistant Sam Yates and visiting scholar Jim O'Keefe. If you want to join the facility, get on the staff, join our Patreon and our Discord, where right now over a thousand nerds are doing their own remote game nights, making special edition facility Magic the Gathering cards and giving me episode ideas, one of which I even got today and I will use. You can go to patreon.com slash Kyle Hill and get your lab coat and join today. And if you support the facility just enough, you get your name on RA here each week. And you can see there's a lot of you, so I don't really know how to pass that. Which boxes would I choose if I was presented with a paradox? Well, I would choose box A and B because no matter what the predictor predicts I will take, I will always be a thousand dollars up, even a thousand dollars or a million plus one thousand dollars. And as the kids say, I could really be using that cheddar paper stacks of cabbage. Oh, still going. Okay. Thanks for watching.