AI's Demand Is Soaring. But Utilities Can't Build Ahead | Joshua Rhodes

With the AI boom, the demand for power is going up significantly. In some cases, like in Texas, the forecasted demand is four times the current grid's capacity. Today we sit down with Joshua Rose, who's a grid expert. He's going to tell us what it takes to build the grid. And will it be in time when the data centers come online? Everything with AI has happened so fast that no one really knows how much we're going to need. But what a lot of grids are doing right now is they're kind of stepping back and they're, you know, tightening the rules around how do you get a large load connected to the system? You know, how much skin do you need to put in the game to make sure that we don't build infrastructure that, you know, goes unused? I'm Siamak. Welcome to Market Insider. Joshua, it's great to have you on. Welcome. Yeah, thanks for having me. Excited to be here. We want to talk to you about data centers and power. You're a grid expert in Texas and you've been studying the grids and how they work, how they operate, and there's a ton of need for power now with data centers. Can you tell us what you're seeing with this demand? Yeah, so the electricity sector is really entering kind of a new phase of growth, as it were, because really since about 2007 or so, electricity demand has been relatively flat across most of the U.S., and some of that has been to, you know, changing of where products and services are made, but some of that is just we've gotten more efficient in how we consume electricity. For a large portion of the country, the electricity sector has been in kind of just kind of a maintenance phase or a maintenance mode. But, you know, a few years ago, we taught the internet how to talk and, you know, built products that allow my students to cheat on their homework, and all of a sudden we have a massive demand for, you know, new data centers to, you know, train and build these new AI models, as well as, you know, other types of data centers to run the models, so for during the inference phase. So that's, you know, when you're asking the model a question. And all of that just takes a tremendous amount of electricity such that it's going to kind of shock the U.S. system out of kind of a maintenance mode and really into a growth mode that a lot of parts of the country, you know, haven't seen. And in Texas, where you're at, and you've been studying Texas too, the demand is going up significantly by multiple folds. Can you tell us about that? Texas is actually one of the exceptions to the rules. We're talking about electricity growth being flat. Texas has been growing for, you know, things like LNG exports and other types of manufacturing facilities, as well as just general population growth. But, you know, Texas is also one of the, you know, one of the most open markets when it comes to electricity, you know, buying and selling. And, you know, that's been attractive to a lot of companies to build generators, to build infrastructure. And, you know, now we're seeing it really attractive to companies to build data centers. To put a little context, there's three main grids in the U.S. You can think of it, the Rocky Mountains East is the interconnection. The Rocky Mountains West is the Western interconnection. But then, you know, famously or infamously, you know, Texas has its own grid. It's called ERCOT or the Electric Reliability Council of Texas. And it actually, Texas actually acts as an island of its own electricity. It's the smallest of the three North American grids. It's about 85,000 megawatts. But if you look at the queue or the list of, you know, large loads, predominantly data centers that want to connect to the system, that list is over 400,000 megawatts long. And so, you know, if all if we built everything that wanted to connect in Texas right now, we'd be looking at, you know, quadrupling or more the size of the of the Texas grid in just a few years. Is that possible based on what you're seeing? Yeah, I mean, that's the well, I guess it's more than the $64,000 question. We're probably at a $64 billion question, I guess, or at this point, to connect all of that is not really not really possible. I mean, you know, we spent 100 years getting the grid, you know, kind of to where to where it is. And, you know, we're not going to, you know, quadruple it in the next, you know, five or six years. And a lot of that is it's not because maybe there's not demand for that electricity or at least, you know, prospective demand for that electricity. It's just that with electricity, you have to match supply and demand in real time. And so all of the electricity that's being consumed for the lights, the computers, the speakers, like everything that's allowing us to talk to each other is actually being generated at a power plant right now. And so if you want to connect more things that take electricity off the grid, you have to connect the same amount of things or more that put electricity on the grid. And then you also have to build the infrastructure that allows that energy to move around. Transformers, transmission lines, all of those, the switch gear, all the other kit that, you know, kind of, you know, allow the system to operate. And, you know, the supply chains for that infrastructure, they're just they've been used to kind of a maintenance phase or a slow growth phase. You know, they're just not set up to be able to, you know, put out as much new infrastructure that would be needed in places like Texas and Virginia and other parts of the US and the world, really. So is there not a lot of storage when it comes to the grid? How does the grid work? With electricity, with the electric grid, you know, got to match supply and demand in real time. There is energy storage on the system, so you can store electricity, say from that's produced by a gas plant or a solar farm or whatever, you can put it into a battery and you can, you know, hold it there and discharge that later. But the magnitude of the amount of storage that we have relative to the amount of energy that is served, you know, on the grid at any given time or, you know, over any given period of time is just pretty small at this point. And so we would just need a lot more storage on the system. But even if you have the storage, you have to have the ability to move that energy around, right? So it's like you could have all the energy in storage, but if you don't have the transmission lines or whatever it is, or the other infrastructure that can move that energy from where it's being put onto the grid to where it's being taken off, you know, that can also be a limiting factor. Now, quadrupling the amount of electricity output from the grid in Texas, for example, how much infrastructure does it take in terms of the transmission lines and the power plants? Yeah, to quadruple the output of the Texas grid, I mean, you're probably looking into the trillions of dollars that it would take to actually realize that. Now, if we were to be able to do that, you know, you would have massive consumption of electricity, and so you'd have, you know, a lot of people paying for that electricity. That doesn't necessarily mean that, like, all that trillions would, you know, go on everyone's electricity bills, like, immediately. But the actual, the capital costs, the capex, like, the infrastructure that would have to be built would, you know, would likely be in the trillions of dollars. Would there be enhancement to the current lines, or do they have to build new lines, build completely new infrastructure? Yeah, so for the linear infrastructure, like transmission lines, you know, sometimes they will only build one circuit on a transmission line, and they've got room on the other side of the poles to put another circuit. And so you can, in some cases, like double the transfer capacity of a particular corridor, or you can upgrade the lines. And so maybe you have lines that are at 69,000 volts, but you can upgrade them to 345,000 volts. And you can, you know, move, you know, five or six times the amount of electricity, you know, across the same corridor. But to serve that much load, you would have to build new infrastructure. So you would have to have new rights-of-ways and new corridors, new power plants, new substations, new everything. We don't really have the technology to just upgrade what we have to be able to, you know, be able to meet that much of an increase in demand. What about the power generation side of things? Do they have to build a lot of infrastructure, or is it not that hard with a gas plant? You can produce a lot of energy, or if there's nuclear or other, we are not building much nuclear, but is it possible to do that in a mega-type power plant? There's nothing that could, that stops, that is stopping us from necessarily building all of that infrastructure. I mean, the only thing that is, is just the supply chains that allow that to be built in a, in the time period that they want it to be built. I mean, we can build all kinds of power plants, and we do. The U.S., you know, right now has about 1,400 gigawatts worth of, you know, worth of power plant capacity, but we would need to add hundreds more gigawatts of capacity to match that demand. Because data centers operate at a pretty high utilization factor, meaning, you know, they may be able to, a lot of times large loads might have an interconnection for, say, 100 megawatts, but they're not very often using close to that 100 megawatts. But data centers, because the compute, the tokens, the models, like everything that they create is so valuable, they want to use as much electricity as they possibly can. And so, you know, we'd have to build the types of power plants that would be able to serve that kind of flat or your slabs of demand. And so that's generally leans towards more thing, more baseload type power plants like, you know, natural gas or nuclear or geothermal, or, you know, renewables plus storage or other types of things like that. And so, yes, we can do it. It would just, I don't know that we can do it on the timeline necessarily that, you know, the current drivers of the system growth want to see. What about the rest of the country? Is it possible to do this in other parts of the country where they have bigger infrastructure? Data centers and power plants can be built in, you know, in other parts of the country. In fact, our highest concentration of data centers are in northeast Virginia. They're close to D.C. and all the three-letter acronym agencies that, you know, that need high levels of compute. And it's something like 70% of the world's internet traffic goes through northern Virginia. It's kind of a wild concept. But there's a lot of data centers that are being built there. There's a lot of data centers that are looking to be built kind of in that same area. They've got great fiber connectivity in other regions. There's other places that might have more room on their grid like North Dakota and other regions. And it is possible to build in those regions. But every single state kind of governs electricity different in kind of how infrastructure gets built, how loads connect to the system. It really can vary from region to region. And, you know, some places are quick, are relatively quick, like Texas, and other places can be, you know, pretty slow, you know, places like, you know, California and other places. And do you anticipate the power costs go up for everybody? That's also, you know, the $64 billion question or another one of those. There's a lot of discussion right now. And there are studies that point both ways on this concept. But if you look across kind of, you know, what are the drivers of, or what are the drivers of electricity cost price growth and the potential drivers for growth? I mean, I think it really comes down to if in a particular region, if the demand for electricity is going, if the pace of the increase in demand for electricity outpaces the supply growth of that electricity, then you're probably going to see prices go up. A lot of places have wholesale markets for electricity. And so they kind of run these auctions to determine the price. And if, you know, supply is greater than demand or is getting, you know, to the higher end of demand, that can drive up the price. That's for the generation of electricity. But then there's also, if you're coming into a region that needs to build a lot of infrastructure to support new data centers, then that can also increase the price of electricity. Because right now, electricity bills, they're roughly around half fixed cost. So paying for the wires, the poles, the things that are, you know, durable, you know, out there. And then about half is, you know, generating that electricity. And so if both of those things get more expensive in any given region, then that can drive up the cost. And you mentioned the way the grid works is we have to have the generation and consumption online at the same time. Is it hard to do this with this amount of power, especially for somewhere like Texas, where you have a significant amount of demand coming online? Is it hard to balance the two? Yeah, I mean, you know, balancing that system is what, you know, the electricity sector has been built to do. Anytime that you flip on a switch and the lights come on, then the system operator is balancing the system correctly. And so for the vast majority of times, you know, we're able to do that. There are some famous examples, like say during winter storm Uri in Texas, where we lost a lot of generation and we're not able to do that. We had to turn people's lights off, which is a situation you definitely, you never want to really get into. But, you know, there are other grids that are, you know, larger than Texas. The PJM interconnection that kind of serves, you know, the mid northeast is, you know, about double the size of ERCOT. And so, you know, it's possible to build much larger grids and still keep them stable. And other grids in places like China are even larger. And so as long as we continue to build, you know, generation at the pace of demand, I believe we have the technology to be able to make sure that we, you know, conserve all of that load. Do you think we can build the generation for the amount of demand we have in the time frame that demand is coming? It's a good question. And it really depends on, you know, how much of this demand is actually real and is actually going to show up. Like in Texas, we've talked about 400,000 megawatts or 400 gigawatts of demand. I don't really believe that much demand is going to be able to show up. And, you know, Texas is putting in structures and processes to try to weed out, you know, what might not be real or what might be, you know, speculative. But this has really been the issue in the electricity sector for the past two years is no one really believes the demand forecast anymore, which is a problem because that's how, you know, companies decide to build new generation or other infrastructure. What a lot of grids are doing right now is they're kind of stepping back and they're, you know, tightening the rules around how do you get a large load connected to the system? You know, how much skin do you need to put in the game to make sure that we don't build infrastructure that, you know, goes unused or isn't prudent to build. But, you know, everything with AI has happened so fast that no one really knows how much we're going to need. But in Texas, we're not going to build 400 gigawatts in that timeframe, but maybe we'll do 40. So that little, you know, it's just... I mean, that's still increasing the grid by 50%. 50% growth. What about these data centers building their own power plants? There's a lot of discussion about them just going directly and because that way they can bypass the need on the community and maybe they can even put them close to the data centers so you don't have to build a lot of infrastructure for transmission, right? Yeah, with a lot of these large loads and the system around this, I mean, it kind of feels like squeezing a balloon right now. And, you know, the balloon pops out in another space. And one of the things you're seeing is a lot of data centers are saying, okay, we'll build our own generation. Like the grid can't come to us, you know, fast enough. And so we're going to build our own generation. But one of the issues with that is, you know, you're competing for the same power plants as the grid's competing for, right? And so it's like, it's not a different type of natural gas power plant that would, you know, serve a behind the meter data center than, you know, just serving the grid. It's the same infrastructure. It's the same transformers. It's the same switch gear. It's all the same infrastructure. And so, you know, you're competing for, you know, for that same infrastructure. And one of the reasons that we have the grid the way we do with a bunch of different power plants, you know, connected to a bunch of different wires is power plants are big, complex machines with a lot of moving parts. You know, things break. If a data center wants to have what we call five nines of reliability, meaning it has power 99.999% of the year, which roughly means you lose power for eight minutes out of the year or something like that. Any individual power plant's only gonna be, you know, maybe mid 90s and 95%, you know, reliable. So if you really wanna get that level of reliability with using your own infrastructure, then you might have to build not one, but two, maybe three, you know, power plants to just make sure that you're competing with the grid and you're competing with everyone else and it, you know, tripled the cost of your power. So you still need to be connected to the grid one way or another. I mean, I think that's what a lot of data centers and, you know, some of these hyperscalers who really haven't been in this hard hardware space, in this hard infrastructure space, realizing that, you know, the multiple redundancies and, you know, reliability impacts of having a massive system that stretches across multiple states in some cases, across, you know, hundreds of thousands of square miles and is able to hook up power plants from far flung regions, just brings a level of reliability that is just really hard to kind of match just by yourself going at it alone. Do you think that there is a limit to the AI growth? Do you think that power could become a limit? Because a lot of people that are in this space, they're coming from the technology and software space. And in software, you can make things fast. But then when it comes to the real world, you know, if there's physical challenges, how do you see it from your perspective? Yeah, I mean, the old adage of, you know, move fast and break things, you know, kind of works well in software, but, you know, it's not going to really work well in an infrastructure like electricity, like water, fiber, and all these other things. I think these hyperscalers and this industry, they're coming to these realities that the infrastructure just, you know, kind of, it moves slower because you have to put real steel in the ground, concrete for footings, for transmission line pylons and poles. And you've got to, you know, get the cables and they've got to be run and they've got to be put in place. And like, that just takes a lot longer to do than some of the timescales that the hyperscalers have been operating in. And, you know, as far as being in the power sector, I mean, I don't think a lot of these, you know, groups want to be in the power sector because the margins are pretty low compared to what they're, you know, used to. I mean, you know, tech margins can be in the tens, tens of percent, 40%, 50%, 60%, you know, margins, you know, margins in the power sector in the single digits. I mean, it's just not, I don't think it's really an industry that they necessarily want to be in. Do you think the power utility companies, do you think they're going to significantly benefit from this AI movement? Or do you see risks there? Given the current structure of the way that we build out most of the infrastructure, I think, you know, a lot of these utilities and power companies do stand to benefit, you know, from this because their sector is moving from kind of a maintenance phase into kind of a growth phase. And a lot of the way that utilities, particularly those that maintain the wires and the poles, you know, make their money is through a regulated rate of return. For every dollar that these utilities spend, they make a guaranteed profit on that money. Now, it's not as high as the tech profits and things like that, because they're guaranteed the profit, but the incentives are set up such that the more you spend, you know, the more you make. And so, you know, if we're going to build a lot of infrastructure, then definitely certain utilities are going to stand to, you know, make a lot of money kind of in that space. But if we need the infrastructure to serve the load, I mean, that's what they exist there to do. What about the rates? Would the rates go up or are they regulated enough that they might not be able to go up as much? Electricity regulation in the U.S. is pretty complicated. And so in some places, utility companies have to go to, you know, state level agencies like public service commissions or public utility commissions to get permission to raise rates because they're allowed to be a regulated natural monopoly in their service area. In some cases like Texas, you can more choose who you get your electricity from and in a handful of other states. And so that's, there's more of a competitive, you know, a competitive space for that to happen. And so if you look across the different regions where you're seeing upward pressure on the price of electricity, it's really in regions that are having trouble, you know, growing the generation and the transmission infrastructure, you know, at the same rate that that demand is coming. If you go to a region that has a lot of spare capacity, then like I'd say in North Dakota right now, then moving more energy, like across the same infrastructure, can actually decrease the costs of electricity. But in certain other places that are more constrained, if you're having to build new infrastructure to support that new, you know, flow of electricity to these new customers and you're still allocating the costs of that infrastructure the same way that you've always done to all customer classes, then yes, you can put upward pressure on electricity rates. Again, you usually have to get public service commissions and public utility commissions to approve that, but you're seeing more approvals than, you know, pushback these days from these bodies. Now, when that demand comes online, does it, is there anybody that's watching this and saying, we can't add another 10 gigawatts because we have customers and we can't, we don't have the power. Are those conversations happening or does the price cut people off? So certain types of customer classes are exposed to price. And so a lot of times you'll have large industrials or large commercial consumers of electricity that may opt to get a real-time price of electricity. But most customers, you know, pay a flat rate that may stay the same for, you know, a year or two. But as the system, you know, changes and grows and more infrastructure is put on there, you know, that there can be pressure, upward pressure put on the price of that. But utilities generally have an obligation to serve. And so, you know, they're going to try to connect as much load as they can. But it is those utilities kind of that are the ones talking to the customers, that are the ones that are interfacing with the customers that really best understand, you know, their local infrastructure to know whether or not that they can, you know, take more of this new load. But a lot of times the rules around assessing how much new load do you know that you can take on are pretty murky. And this is one of the things that the Federal Energy Regulatory Commission has, you know, recently directed a lot of these utilities and a lot of these independent system operators, grid operators to really figure out, telling them to streamline their processes and make this more transparent. Because it's hard for them to know if they can service X amount more or not, right? Right, because of the same network structure that allows, you know, different power plants to, from, you know, far away to be connected to the same system and put energy on the same system. If I'm a large load, if I'm a data center and I wanna connect in a certain part of the network, I'm, and there's another large load that wants to connect, you know, not too far away from me. You know, the fact, maybe one of us could connect, but both of us connecting might destabilize the system. And so the concept of who comes first, who gets a connection, who doesn't, you know, there's all of these rules that we're now having to work out because, you know, previously these large loads would come at an infrequent enough pace that we could just kind of take them as they came. But now if you're getting, you know, dozens of requests or in some cases, hundreds of requests at the same time, you just can't possibly do that, you know, one at a time because, you know, data center A is affected by data center B, large load C, et cetera. And so, you know, we've, in Texas where we've got this thing called the batch zero process that we're doing to try to figure this out. And, you know, FERC is also directing other parts of the country to do this in a more disciplined way. So do you think in this process the prices that the data centers will have to pay for power might go up? Or do you think it would be still this, they will get treated still as a normal customer, but they just consume more power? Or is it going to be that they will get a special pricing to get connected? If there's 10 of them that want to get connected and two of them happen to be. One of the conversations that is being had across, you know, the entire country from the federal level down to the state level is should we be treating these new large loads in particular data centers as a separate class of customer? One that maybe we don't offer the highest level of reliability or that we, you know, make pay for more of the infrastructure that they cause. And the reason that that conversation is coming around is that the pace at which these data centers are looking to connect to the system is such a higher velocity than we've seen in the past that if we build a lot of infrastructure that is just going to one single customer class, you know, then maybe that customer class should be the one to pay for that infrastructure. And while that sounds nice and easy to do or it sounds like a good idea in a lot of different places, sometimes the regulatory constructs are that the data center can't do that. Like the rules and regulations in place don't allow that to happen. But a lot of state, there's a lot of political, you know, wins that are, you know, pushing state legislators and federal legislatures to change those regulations to allow, you know, these companies that, you know, appear to have a lot of money and are willing to spend a lot of money, you know, maybe they can spend some of that money in the infrastructure side, you know, not just building, you know, the data center side. Now, Joshua, we will be watching this closely and we'll touch base with you on a regular basis. Joshua Rhodes, it was great to have you on Market Insider. Thanks for having me. Tell us what you think about this conversation with Joshua. Are you concerned about our power supply going forward, especially as we compete with these data centers as they come online? Make sure to tell us in the comments. I'm Siamak, this is Market Insider. We'll see you next time.