Why No One Can Slow Down AI (Game Theory Explains It)

So there's all this talk out there about, oh, why don't we just slow AI down? Let's sign an agreement as a world, as a country, and let's pace the frontier. Let's slow the leading models and companies down, and that will help to regulate some of the risk of AI. And the reason that's usually cited for why that can't happen is greed, like, oh, the companies want to just make more money. I want to tell you. No. Maybe that's part of it, right? I'm not going to speak on behalf of those companies or people, but what it comes down to is the math of it and the fact that rational individuals and companies and countries playing this game of AI are going to keep because the math shows that it is in their best interest to do so. That is the game theory of AI and frontier development today. And in this In this video, I want to give you a high-level overview on what that looks like. And how this is something that we've seen 100 times before play out throughout history and specifically with nuclear development. Now, there is a bright side there in that, hey, nuclear arms have not proliferated everywhere. So you can, in theory, do the same for AI. So we'll touch on that as well. And then at the end, we'll also look at the AI strategy of China. And so this video is part of the AI Fundamentals series. This is. The Game Theory of AI. Let's hear a word from the sponsor of today's video, and then we'll get into it. Now speaking of, I want to show you something that I set up for my own business. So on the channel, we have a cold brew brand called Alto. And the challenge there is answering the same customer emails over and over again. So I built an AI agent that does it for me. And we did that with Make, who's the sponsor of this part of the video. And Make lets you connect over 3000 apps. And now there's AI agents that can make decisions for you inside of those workflows. So you don't have to build anything by hand. And you can just talk to my... Maya, which is makes AI assistant. So I can say something like this. So build an AI agent for my coffee brand when a customer email arrives in Gmail, Find their order. If they want a refund, send a Slack message. And if I approve it, reply to the customer. And here we can see it finds the order, it checks the policy, it sees that this one is qualified. And then it didn't reply yet, so it asked me first, I approved it, and then finally it can send that reply. So if you answer the same emails all day, you can build this and a lot more than this on Make. So links in the description, and just comment what you want to Maya. That's make.com again, or link in the description. So in 1950, political scientist named John Firth named a problem that he called the security dilemma. And the setup is pretty simple. It says, okay, there's no world peace. So we have a set of countries and each country must defend itself. But the tools that make me safe as a country make you less safe because a defensive weapon is also an offensive weapon, right? But if I build a stronger army to defend myself... You can't read my mind. You only see me building a bigger army, so you build one too. And now I see your bigger army, so I build an even bigger army. Nobody wants a war, nobody wants an arms race or having to spend more money on defense, but both of you end up poorer and less safe than when you started. And in 1976, this was turned into the spiral model. And the core insight there is that conflict does not necessitate an aggressor. You can have two defensive. players or countries or individuals, whatever you want to call them, and as long as there's uncertainty in the mix about defensive versus offensive measures and capabilities and intention, then that can spiral into conflict. And this is very fundamentally a cooperation problem. This is endemic to game theory. If you study this stuff, it applies to a lot of games. And one of the extremely material ways in which we see that manifest is through arms races and conflict and nuclear proliferation, and then now finally, AI. And so let's just map that security problem, so to speak, onto AI. If I know that another country can use AI to attack my systems, then defending against That level of intelligence or that level of intelligent model requires at least an equal or greater degree of intelligent models on my end. Therefore, unless I leave myself and my citizens vulnerable, I should support building up those defensive capabilities. But now what I'm doing is building frontier AI. And now the other country will look at me and say, hey, they just built an AI system just as good or better than ours. Now I'm vulnerable, right? Right. And then they will build out the frontier intelligence and then you just. Tier up, right? You just lever up in that game and so these systems get more and more intelligent. And here's what makes this even worse for AI, right? With nuclear proliferation, defense and offense at least look different. If someone is building a bunch of nuclear weapons versus a missile shield versus a set of bunkers, one of those is clearly more offensive in nature as in you can't physically use a nuclear weapon for defense. And then vice versa, you can't use a bunker for offense, at least materially. But with AI, at least in the current iterations, they are the same object. The best frontier defense model is also the best frontier offense model in terms of its cyber capabilities, its ability to manipulate, to operate. And there was a 1970 paper called Cooperation Under the Security Dilemma. Where they said, two variables decide how dangerous a security dilemma of this sort is. And the first question there is, can you distinguish offensive weapons from defensive ones? In this case, the answer is, at least currently, no. If you could, that would make cooperation or an agreement a lot easier because it's enforceable, right? You can tell if someone is building something purely for offensive purposes. And then the second question that was proposed was, Okay, does offense or defense have the advantage? If defense is cheaper, that's a lot better. But in this case, again, since the object is indistinguishable, then the cost to develop those offense and defense measures are the same, because the tool itself is the same. There is some ongoing debate there. You can make the argument that cyber favors offense, since if you're defense in that scenario, you have to close all the gaps. If you're offense, you just need to find one hole. Others argue that that doesn't really matter because of how fast these models work. So whoever is the best model just has the advantage regardless, like I just said. We'll see how that plays out. So that all said, before we go any farther, I don't want to make this sound like it's this really bad situation. If you look at arms races over the past 50 years, those things have not spiraled out of control. So clearly there are measures. And again, we're going to get to that. But first, what I want to do is just kind of write out the game so we can visualize this more materially. So let's say we have two players. You could say it's the US or China. Let's just call it Lab A and Lab B. And each has two strategies. You can restrain or you can race. And the question that's gonna matter here is the order of the payoff. Now, if you're familiar with the Prisoner's Dilemma, this is going to look very familiar to you. Version one here is the Prisoner's Dilemma, whereas if we both restrain, we get three. If we both race or try to make the best model, we each get one. But if I develop these amazing models and you don't, right, then I have this massive security advantage over you. I get five and you get zero. And so the equilibrium here is 1-1. And if you're not at all familiar with the Prisoner s Dilemma, go watch a video on it. It's really interesting stuff. And I'll say that again. The predicted outcome in this scenario is 1,1, e.g. a payoff of one point or unit, whatever you want to call it, for both players. Even though... 3-3 was sitting on the table. We both could have got a better outcome, but we didn't. We both chose to race and develop all of these models and spend more on defense, even though we didn't have to. Because mathematically, if you are rational and these are the payoffs, then that's what you're going to do. That's what you have to do. That is, that's what you have to do if the payoffs are like so. But now if we look at the game on the right here, this is a little bit different in that it pays the most to both restrain. And here the problem just comes down to trust. Clearly both people would be better off if you can be restrained and vice versa. So the problem isn't do we want to cooperate, it's do we believe the other party will. So which of those two games is AI? Is it an incentive problem or is it an assurance and a trust problem? That depends on the belief of what is the value of being first in the domain of artificial intelligence. First as in first to super intelligence, first to just these crazy cyber, physical, digital capabilities. Well, most people think that being first gives you a very material advantage, a major advantage over competition, both from the perspective of a company developing the model and a country which has that capability. And if that's what you believe, then you can very easily see how racing, so to speak, beats restraint. It's just the payoffs are better for you and your constituents. And if that's the case, the way out is enforcement. You need some mechanism to be able to enforce or pace frontier model development. And that's where we get to an even tougher problem with AI is, okay, let's take it face value. We need enforcement. A mechanism to enforce if there was going to be an agreement to cooperate or to restrain. And how exactly do you plan on doing that with artificial intelligence? No one, and I mean no one, has figured that out at the global scale. And if you can't enforce, then you can't cooperate. And if you can't cooperate, then you're going to get that payoff, which is, at least in that setup, worse for everyone. Now, this in mind. There was the Cold War, there was the nuclear arms race, and that worked out. So why can't we just copy that playbook? And this goes back to what I just said about AI and kind of the enforcement problem. So let's just walk through the nuclear game and the features that had, which made cooperation possible versus AI. Now, first is the structure of the prize. So economists call a race like this an all-paid. Contacts, so every player pays to compete, but only one wins. That's, think of it like, you know, a patent race, for example. So in an all-pay contest, the bigger the prize, the more everyone spends, and the total spending can eat most of the prize's value. And if people believe AI's prize is close to everything, the spending has no natural ceiling. Basically, if you believe this thing is super, super valuable, There is no limit to how much you're going to spend, even if that cumulative spending on it then erodes the actual on-paper value of the prize, but it doesn't for the winner. And there's a model of this exactly for AI. In 2016, Armstrong, Bostrom, and Shulman modeled teams that were racing to build advanced artificial intelligence. And each team chooses how much to spend on safety, but safety costs speed. And their result was that the more teams you had in the race, The more they distrust each other and the more safety gets cut out. So close races are the most dangerous. When you think you are one step behind, right, you are going to skip the brakes. And so that structure of the prize is very different than in the nuclear case because nuclear has much less of a positive outcome, right? It is less net valuable to society except as a. purely defensive or offensive weapon versus the general utility of super intelligent models. Now, second is verification, right? So Cold War arms deals worked because you could just count. Satellites could see the silos. Inspectors could count warheads. And again, you just have this verification or this enforcement problem that's really severe when you come to this world of models. And so we looked at that stag hunt game, right? If the AI game is an assurance problem. Verification would be the cure. But if we can't do that, then it slides into this just racing, essentially, that worse outcome for everyone. And then third is a commitment spread. So in 1995, James Fearon asked why rational states ever fight when war is so costly. And one of his answers was the commitment problem. So when power shifts, right, the rising side cannot credibly promise to stay restrained later if we have a dominant power and a rising power. It's the Thucydides trap problem, basically. And the side that is declining knows it. So the declining side has a reason to act now while it still can. And AI is a machine that is built for fast shifts in power, right? A lead can open or close in months. And that makes every promise of restraint that much harder to believe, because that could mean the difference between leading or not leading again in this very important race from the perspective of those players and then fourth is the number of players so the nuclear club stayed pretty small because bombs are just hard to build right and enriched uranium is kind of a physical bottleneck in all of this AI capability is super distributed anyone can download a model onto their computer on their CPU the basic situation is last year frontier's capability are becoming this year's open models and that means the game does not stay You know, a two-player model becomes an n-player model, and as n grows, or the number of players, cooperation gets much, much harder, and more players means more chances that a single player defects, and the math really starts to favor not cooperating the more there's a chance that any single player out of that string of n players defects. And then fifth and finally, this kind of a game is nested. It's kind of like a two-level game where states may negotiate with each other, but inside each state, labs are beating or competing with each other for talent and chips and customers. So that stacks the dilemma where you have the state-level dilemma, and then also internally within each country, you have the lab-level dilemma or the corporate dilemma. That is not a setup you had in the case of nuclear, where it was purely state-funded arms. So if we put all of that together to level set. We have a huge prize, no ways to verify or enforce power that can shift very fast, many players and two levels. Those are five things that apply to that game theory of AI world that do not apply to nuclear. Now, you are seeing all of this play out in the real world in real time. Anthropic Mythos, for example, was first released as a defense tool under Project Glasswing. Then there were export controls put on that technology. And then I just made a video on critical minerals, and we made another one on energy. And you can basically think of AI as being bottlenecked by those critical minerals or compute or energies, just whatever is upstream in the supply chain. And both at the state level, both players, the U.S. and China, are using those as political leverage against one another. Basically, they are acting in accordance with the principles laid out prior in this video. And now before we get to the positive side of this, which is the ways out of what I have so far painted as a pretty difficult game for cooperation, I just want to say that the reason that China specifically or Chinese labs are publishing so many open source models It's because they're marginally behind. And when you're behind, as a rule, you go open source. A free model that is nearly as good as a leading model will really, really, really harm the leading model and will not harm you nearly as much. You'll get more users even if what you're offering is free. And what you're doing as the laggard in that situation if you're behind is you're giving up revenue, right, to earn market share. Something like 96% of revenue in AI goes toward the closed source. labs or models, but it's something like about half of usage or tokens is going through open models. And so basically the strategy in AI, just as with a lot of other things is one of those strategies is if you're behind open source, your model. And so that's what you're seeing play out at the massive global level, not necessarily because it's a state thing, but just because if I'm a company in China and I know the US models are six months or a year ahead of me or whatever I'm going to just try and do something much cheaper and open source it than what those guys are doing to gain market share such that in the future, if I catch up, then I can start to charge for it and make a bunch of money. This said, let's jump back to how we can cooperate globally or if there was to be some kind of global agreement to pace the frontier, how you could, from this same perspective we've used in this video, make that happen. So first, the way that you get out of the prisoner's dilemma is Pretty often you assume that you don't just play it once, right? This is a repeated game. There is debate as to whether that applies to AI, but let's just say that actually we're playing a prisoner's dilemma geopolitically every year for the next thousand years. If that's the case, then I can punish you tomorrow for cheating or defecting today. There are strategies like tit for tat that kind of get you out of there, a bunch of other ones. But basically, if you do value the future, Then you can't usually make an agreement like that work. We still run into those enforcement problems, but generally if there are clear signs that someone is, and this really matters, if there are clear signs that someone is cheating, then you can punish them in the future for it, and if there's enough games, then, you know, the math works out and everyone will cooperate. You still have the pretty gnarly enforcement problem, but... That is kind of a path through. And then it's just a tech problem. Then it's just a question of, OK, can we figure out how to see who's building really advanced models? And so one way of doing that, for example, is trying to go for a proxy like the number of GPUs you have. If we assume that frontier intelligence requires a lot of GPUs, then you can roughly track the volume of GPUs people have if you're talking tens of thousands, hundreds of thousands plus. So if we can work as a world to make enforcement or verification possible, then that opens up that first route of, okay, I can now punish people for cheating. This is a repeated game. And now based on that, we can cooperate. Now I think second, we are going to see a lot more effort put into defense across the board. Like AI, you saw this with Mythos and Glasswing. You're going to see that at massive scale moving forward. And especially cyber defense as the first domain there is going to be a really big, I think, product service suite for AI moving forward. And now finally, I want to end on this, right? How you think about the payoffs really matters here. If you believe that the only safe thing for you to do is to win this race, that's one framing. Another framing is the race itself is the danger. And if you get into mutually assured destruction territory, if you assume that the race itself is so much of a danger, that changes the map. And if there is enough conviction in that, excuse me, there's not currently an AI, but if there was globally, then people would start to act differently and you could have this really strong internal incentive to restrain. And if everyone has that similarly strong incentive, If everyone really believes that the race itself is so dangerous to them that they're better off doing anything except continuing to play that race, then you're good and you can all cooperate because enforcement, essentially, verification is done internally. And so there are ways out of this if you do want global enforcement of kind of pacing the frontier. I think what it comes down to and just. a big picture sense is there's a lot of uncertainty remaining. People disagree on how valuable, how dangerous, how competitive these technologies are, and that creates really a very unleveled playing field for figuring out a way to cooperate from. If everyone had the exact same information and believed the same things, it would be much easier, but that's just not at all the case. And again, you only start to ask these questions in the first place. Based on your belief about AI, you only start to ask, hey, should we pace the frontier globally if you're really concerned about the potential dangers of the technology? And I want to wrap it up on that note. I hope the dynamics here are interesting to you and food for thought. Of course, specifics are going to be disagreed upon, but I tried to make this or frame it as just a high level. These are roughly the situations, the games, the incentives that people are thinking about. And I want to restate one of the really fundamental lessons from game theory, which is you don't need villains. You don't need people acting in bad faith or acting maliciously to create a bad outcome. Perfectly rational individuals, companies, states who are acting in the best interest of the citizens that they represent can create a worse outcome for everyone. The way out of that is not to change the players. And generally, people are really good actors. They act in good faith. They want good things on behalf of their constituents. The way out, if you do want to create a better outcome for everyone, is to change the structure of the game. And so if we as a world decide we want to pace the frontier of AI, that's what needs to happen. You have to change the structure of the game. And if you can do that, you can cooperate. If you don't do that. You can be acting in good faith on both sides, and you can still never get a cooperative agreement. And so we started this video on that question of, well, why hasn't global AI pacing the frontier agreements happened? There you go. The game needs to change. I'll wrap it on that note. Appreciate you being here, and I will see you in the next one. Cheers.