Tesla AI Day in 23 Minutes (Supercut) (2022)

Welcome to Tesla AI Day 2022. We've got some really exciting things to show you. I think you'll be pretty impressed. I do want to set some expectations with respect to our Optimus robot. As you know, last year, it was just a person in a robot suit. But now we've come a long way. And compared to that, it's going to be very impressive. We're going to talk a lot about our progress in AI, Autopilot, as well as the progress with Dojo. So should we bring up the bot? Before we do that, we have one little bonus tip for the day. This is actually the first time we try this robot without any backup support, cranes, mechanical mechanisms, no cables, nothing. We'll show you some videos now of the robot doing a bunch of other things. Yeah, we wanted to show a little bit more what we've done over the past few months with the bot, and just walking around and dancing on stage. Just humble beginnings, but you can see the Autopilot neural networks running as is, just retrained for the bot, directly on that new platform. That's my watering can. Yeah, when you see a rendered view, that's the robot, that's the world the robot sees. So it's very clearly identifying objects, like this is the object it should pick up. We use the same process as we did for Autopilot to collect data and train neural networks that we then deploy on the robot. That's an example that illustrates the upper body a little bit more, something that we'll try to nail down in a few months, over the next few months, I would say, to perfection. That's not the only thing we have to show today, right? Yeah, absolutely. So what you saw was what we call BumbleSee. That's our rough development robot using semi-off-the-shelf actuators. But we actually have gone a step further than that already. The team's done an incredible job. And we actually have an Optimus bot with fully Tesla-designed and built actuators, battery pack, control system, everything. It wasn't quite ready to walk, but I think it will walk in a few weeks. But we wanted to show you the robot, something that's actually fairly close to what will go into production, and show you all the things it can do. So let's bring it out. So here you're seeing Optimus with the degrees of freedom that we expect to have in Optimus production unit one, which is the ability to move all the fingers independently, to have the thumb have two degrees of freedom, so it has opposable thumbs, and both left and right hand. So it's able to operate tools and do useful things. Our goal is to make a useful humanoid robot as quickly as possible. And we've also designed it using the same discipline that we use in designing the car, which is to design it for manufacturing, such that it's possible to make the robot in high volume at low cost, with higher reliability. Optimus is designed to be an extremely capable robot, but made in very high volume, probably ultimately millions of units. And it is expected to cost much less than a car. So I would say probably less than $20,000 would be my guess. The potential, like I say, really boggles the mind. Because you have to say, what is an economy? An economy is sort of productive entities times their productivity. Capita times productivity per capita. At the point at which there is not a limitation on capita, it's not clear what an economy even means at that point. An economy becomes quasi-infinite. This means a future of abundance, a future where there is no poverty, where people, you can have whatever you want, in terms of products and services. It really is a fundamental transformation of civilization as we know it. It's very important that the corporate entity that makes this happen is something that the public can properly influence. And so I think the Tesla structure is ideal for that. All right. So you've seen a couple robots today. Let's do a quick timeline recap. So that robot that came out and did the little routine for you guys, we had that within six months. Built, working on software integration, hardware upgrades over the months since then. But in parallel, we've also been designing the next generation. This one over here. So this guy is rooted in the foundation of the vehicle design process. We're leveraging all of those learnings that we already have. Again, we're using that vehicle design foundation. So we're taking it from concept through design and analysis, and then build and validation. Along the way, we're going to optimize for things like cost and efficiency, because those are critical metrics to take this product to scale eventually. So in the middle of our torso, actually it is the torso, we have our battery pack. This is sized at 2.3 kilowatt hours, which is perfect for about a full day's worth of work. So going on to sort of our brain. It's not in the head, but it's pretty close. Also in our torso, we have our central computer. So as you know, Tesla already ships full self-driving computers in every vehicle we produce. We want to leverage both the autopilot hardware and the software for the humanoid platform. But because it's different in requirements and in form factor, we're going to change a few things first. So we still are going to do everything that a human brain does. Processing vision data, making split-second decisions based on multiple sensory inputs, and also communication. So to support communications, it's equipped with wireless connectivity as well as audio support. And then it also has hardware level security features, which are important to protect both the robot and the people around the robot. So can we utilize our capabilities and our methods from the automotive side to influence a robot? And since we had crash software, we're using the same software here, we can make it fall down. The purpose of this is to make sure that if it falls down, ideally it doesn't, but it's superficial damage. So we wanted to dust itself off, get on with the job it's being given. So our actuator is able to lift a half-ton, nine-foot concert grand piano. For our robotic hand design, we were inspired by biology. We have five fingers and a posable thumb. Our fingers are driven by metallic pendants that are both flexible and strong. We have the ability to complete wide aperture power grasps while also being optimized for precision gripping of small, thin, and delicate objects. All right, so all those cool things we've shown earlier in the video were possible just in a matter of a few months thanks to the amazing work that we've done on autopilot over the past few years. Most of those components ported quite easily over to the bots environment. If you think about it, we're just moving from a robot on wheels to a robot on legs. So some of those components are pretty similar, and some other require more heavy lifting. So for example, our computer vision neural networks were ported directly from autopilot to the bots situation. We're also trying to find ways to improve those occupancy networks using work made on neural radiance fields to get really great volumetric rendering of the bots environment. For example here, some machinery that the bot might have to interact with. Another interesting problem to think about is in indoor environments, mostly with absence of GPS signal, how do you get the bot to navigate to its destination? So we've been training more neural networks to identify high frequency features, key points within the bots camera streams, and track them across frame over time as the bot navigates with its environment. And we're using those points to get a better estimate of the bot's pose and trajectory within its environment as it's walking. And this is a video of the motion control code running in the autopilot simulator showing the evolution of the robot's walk over time. So as you can see, we started quite slowly in April, and started accelerating as we unlocked more joints and deeper and more advanced techniques like arms balancing over the past few months. Right, so hopefully by now you guys got a good idea of what we've been up to over the past few months. We started with something that's usable, but it's far from being usable. There's still a long and exciting road ahead of us. I think the first thing within the next few weeks is to get Optimus at least up to par with BumbleSee, the other bot prototype you saw earlier, and probably beyond. We're also going to start focusing on the real use case at one of our factories, and really going to try to nail this down and iron out all the elements needed to deploy this product in the real world. I was mentioning earlier, indoor navigation, graceful form management, or even servicing, all components needed to scale this product up. But I don't know about you, but after seeing what we've shown tonight, I'm pretty sure we can get this done within the next few months or years, and make this product a reality and change the entire economy. So I would like to thank the entire Optimus team for all their hard work over the past few months. I think it's pretty amazing. All of this was done in barely six or eight months. Thank you very much. Hi, I'm Ashok. I lead the Autopilot team alongside Milan. This time around last year, we had roughly 2,000 cars driving our FSD beta software. Since then, we have significantly improved the software's robustness and capability, that we have now shipped it to 160,000 customers as of today. For example, we trained 75,000 neural network models just last one year. That's roughly a model every eight minutes coming out of the team, and then we evaluate them on our large clusters, and then we ship 281 of those models that actually improve the performance of the car. And this pace of innovation is happening throughout the stack. The planning software, the infrastructure, the tools, even hiring, everything is progressing to the next level. Let's use this intersection scenario to dive straight into how we do the planning and decision making in Autopilot. So we are approaching this intersection from a side street, and we have to yield to all the crossing vehicles. Right as we are about to enter the intersection, the pedestrian on the other side of the intersection decides to cross the road without a crosswalk. Now, we need to yield to this pedestrian, yield to the vehicles from the right, and also understand the relation between the pedestrian and the vehicle on the other side of the intersection. A lot of these intra-object dependencies that we need to resolve in a quick glance. And humans are really good at this. We look at a scene, understand all the possible interactions, evaluate the most promising ones, and generally end up choosing a reasonable one. But the same framework extends to objects behind occlusions. We use the video feed from eight cameras to generate the 3D occupancy of the world. The blue mask here corresponds to the visibility region, we call it. It basically gets blocked at the first occlusion you see in the scene. We consume this visibility mask to generate what we call as ghost objects, which you can see on the top left. Now, if you model the spawn regions and the state transitions of these ghost objects correctly, if you tune your control response as a function of their existence likelihood, you can extract some really nice human-like behaviors. Now I'll pass it on to Phil to describe more on how we generate this occupancy network. The occupancy network takes video streams of all our eight cameras as input, produces a single unified volumetric occupancy in vector space directly. For every 3D location around our car, it predicts the probability of that location being occupied or not. Let's talk about some training infrastructure. So we've seen a couple videos, there's no four or five. I think and care more and worry more about a lot more clips than that. So we've been looking at the occupancy networks just from Phil. Just Phil's videos, it takes 1.4 billion frames to train that network that you just saw. And if you have 100,000 GPUs, it would take one hour, but if you have one GPU, it would take 100,000 hours. So that is not a humane time period that you can wait for your training job to run, right? We wanna ship faster than that. So that means you're going to need to go parallel. So you need more compute for that. That means you're going to need a supercomputer. So this is why we've built in-house three supercomputers comprising of 14,000 GPUs, where we use 10,000 GPUs for training and around 4,000 GPUs for auto-labeling. So I could go on and on. I just went on, touched on two projects that we have internally. But this is actually part of a huge continuous effort to optimize the compute that we have in-house. So accumulating and aggregating through all these optimizations, we now train our occupancy networks twice as fast just because it's twice as efficient. And now if we add in a bunch more compute and go parallel, we can now train this in hours instead of days. And with that, I'd like to hand it off to the biggest user of compute, John. Hi, everybody. My name is John Emmons. I lead the Autopilot Vision team. I'm going to cover two topics with you today. The first is how we predict lanes, and the second is how we predict the future behavior of other agents on the road. All right, so ultimately what we get from this lane detection network is a set of lanes and their connectivities, which comes directly from the network. There's no additional step here for sparsifying these dense predictions into sparse ones. This is just the direct unfiltered output of the network. Okay, so I talked a little bit about lanes. I'm going to briefly touch on how we model and predict the future paths and other semantics on objects. So I'm just going to go really quickly through two examples. The video on the right here, we've got a car that's actually running a red light and turning in front of us. What we do to handle situations like this is we predict a set of short time horizon future trajectories on all objects. We can use these to anticipate the dangerous situation here and apply whatever braking and steering action is required to avoid a collision. So putting it all together, the Autopilot Vision Stack predicts more than just the geometry and kinematics of the world. It also predicts a rich set of semantics, which enables safe and human-like driving. Let's talk about auto-labeling. So we have several kinds of auto-labeling framework to support various types of networks. But today, I'd like to focus on the awesome lanes net here. This machine easily scales as long as we have available compute and trip data. So about 50 trips were newly auto-labeled from this scene, and some of them are shown here. So 53 from different people. This is how we capture and transform the space-time slices of the world into the network supervision. Take, for example, the simulated scene playing behind me. A complex intersection from Market Street in San Francisco would take two weeks for artists to complete. And for us, that is painfully slow. However, I'm gonna talk about using Jagen's automated ground truth labels along with some brand new tooling that allows us to procedurally generate this scene and many like it in just five minutes. That's an amazing 1,000 times faster than before. And this really sets us up for size and scale. As you can see on the map behind us, we can easily generate most of San Francisco's city streets. And this didn't take years or even months of work, but rather two weeks by one person. And now, to come full circle, because we generated all these tile sets from ground truth data, they contain all the weird intricacies from the real world, and we can combine that with the procedural, visual, and traffic variety to create limitless targeted data for the network to learn from. And that concludes the sim section. I'll pass it to Kate to talk about how we can use all this data to improve Autopilot. Thank you. This data engine framework applies to all our signals, whether they're 3D multicam video, whether the data is human-labeled, auto-labeled, or simulated, whether it's an offline model or an online model. And Tesla's able to do this at scale because of the fleet advantage, the infra that our engine has built, and the labeling resources that feed our network. To train on all this data, we need a massive amount of compute, so I'll hand it off to Pete and Ganesh to talk about the Dojo supercomputing platform. I'm frequently asked, why is a car company building a supercomputer for training? And this question fundamentally misunderstands the nature of Tesla. At its heart, Tesla is a hardcore technology company. Tonight, we're gonna talk a little bit about Dojo and give you an update on what we've been able to do over the last year. Now, last year, we showcased our first functional training tile. And at that time, we already had workloads running on it. And since then, the team here has been working hard and diligently to deploy this at scale. Now, we've made amazing progress and had a lot of milestones along the way. And of course, we've had a lot of unexpected challenges. But this is where our fail fast philosophy has allowed us to push our boundaries. Now, by focusing on density at every level, we can realize the vision of a single accelerator. Now, starting with the uniform nodes on our custom D1 die, we can connect them together in our fully integrated training tile. And then finally, seamlessly connecting them across cabinet boundaries to form our Dojo accelerator. And altogether, we can house two full accelerators in our exit pod for a combined one exit flop of ML compute. Now, altogether, this amount of technology and integration has only ever been done a couple of times in the history of compute. Next, we'll see how software can leverage this to accelerate their performance. Dialogical reduction followed by global reduction towards the middle of the tie, then the reduced value broadcast radiating from the middle, accelerated by the hardware's broadcast facility. This operation takes only five microseconds on 25 Dojo dies. The same operation takes 150 microseconds on 24 GPUs. This is an orders of magnitude improvement over GPUs. So how do we do on these two networks? The results we're about to see were measured on multi-die systems for both the GPU and Dojo, but normalized to per-die numbers. On our auto-labeling network, we're already able to surpass the performance of an A100 with our current hardware running on our older generation VRMs. On our production hardware with our newer VRMs, that translates to doubling the throughput of an A100. And our model showed that with some key compiler optimizations, we could get to more than 3x the performance of an A100. We see even bigger leaps on the occupancy network, almost 3x with our production hardware, with room for more. And this Dojo tile costs less than one of these GPU boxes. What it really means is that networks that took more than a month to train now take less than a week. So we started with hardware design that breaks through traditional integration boundaries in service of our vision of a single giant accelerator. We've seen how the compiler and ingest layers build on top of that hardware. So after proving our performance on these complex real-world networks, we knew what our first large-scale deployment would target, our high arithmetic intensity auto-labeling networks. Today, that occupies 4,000 GPUs over 72 GPU racks. With our dense compute and our high performance, we expect to provide the same throughput with just four Dojo cabinets. And these four Dojo cabinets will be part of our first Exapot that we plan to build by quarter one of 2023. This will more than double Tesla's auto-labeling capacity. The first Exapot is part of a total of seven Exapots that we plan to build in Palo Alto right here across the wall. And we have a display cabinet from one of these Exapots for everyone to look at. Six tiles densely packed on a tray, 54 petaflops of compute, 640 gigabytes of high bandwidth memory, with power and host defeated. We really wanted to show the depth and breadth of Tesla in artificial intelligence, compute hardware, robotics, actuators, and try to really shift the perception of the company away from, you know, a lot of people think we're like just a car company or we make cool cars, whatever, but they don't have, most people have no idea that Tesla is arguably the leader in real world AI hardware and software. And that we're building what is arguably the most radical computer architecture since the Crayon supercomputer. And I think if you're interested in developing some of the most advanced technology in the world that's gonna really affect the world in a positive way, Tesla's the place to be.