DeepSeek‑r1, o3‑mini & Gemini Flash 2.0: Which model should you use in your AI Agents?

Right now the AI world is on fire with new models dropping left and right and in today's video we're going to figure out which of these new models makes for the best AI agents. As of 24 hours ago, Google just released their Gemini Flash 2.0 model that has an insane 1 million token context window and is somehow even cheaper than DeepSeek R1. And a few days before that, OpenAI just released their O3 mini model, which looks like it's the best developer model out there because it's insanely smart and super efficient. And in this video, we're going to put all these models to the test to figure out which one makes for the best AI agent by comparing them against the most important task AI agents need to perform. And to keep things interesting, we're also going to throw some of the previous best models out there like CloudSonic 3.5 and GPT 4.0 to see how these old models stack up against the new ones. And because you guys are awesome, I'm going to be giving away all the source code in this video completely for free, which does include one of my favorite AI agents that I've ever built. Just check out the description below for more information. But enough of that, let's go ahead and dive into the video and start testing out all these models to figure out which one makes the best AI agent. And real quick, before we just dive into testing, we need to do a quick breakdown of all of these models to figure out which one should be the best AI agent, at least on paper. And we're going to do that by comparing the five core stats of what goes into making a great AI agent. So right out the gate, Gemini 2.0 Flash, the newest model has an insane context window, 1 million tokens. So at this point, that means you can throw multiple books worth of knowledge in here and pretty much skip all of RAG because it can handle so much information. What's also important to note is it's pretty cheap, but it does have a limited output tokens. Next, we got O3 mini, which is taking over the world by storm because it's pretty balanced. It has a 200,000 input token and it has a 100,000 output token limit, which is the biggest right now. So at this point, if you're trying to generate a lot of content, like a small book or a bunch of newsletters, this is your model to go to. And it's also a really good developer. Next, we have DeepSeek R1, which pretty much broke the internet two weeks ago because it's open source. It's the cheapest model to use right now, and it's also the smartest. So all around, it has a lot of pros. I also do want to do a quick throwback to one of the most commonly used models by developers, Cloud 3.5 Sonnet. At this point, it is one of the most widely used models by AI developers, but it has a huge negative because it is insanely expensive for what you're getting at this day and age. So at this point, it's seven times more expensive than DeepSeek. So at this point, I've cut it out of my development lifecycle. Next, we have GP2 4.0, which has pretty much been the standard for all agents. But at this point, it's not looking so good when you compare it to the rest of the models that are available. So at this point, on paper, it looks like O3 Mini should be the best model, at least on paper. But enough talking about it. Let's go ahead and start putting these models to the test and figure out which one makes for the best AI agent. So let's go ahead and dive into test number one. All right, guys. So it's time to go ahead and dive into our first test, which is called Instruction Overload. And in this test, what we want to do is make sure that these models can properly follow instructions. So to test them, what we're going to do is use these agents to go off and write a newsletter, but in a very specific format with a ton of different rules to really put these agents to the test. Then once we're done testing all these different models, we're going to compare them to see how long they took to run and complete the task, how many tokens were used, the overall price, and most importantly, did they pass or fail? And then once we've done that, we'll actually do a quick deep dive to actually look at a side-by-side comparison of all the final generated newsletters so that we can see how they stack up against each other to see which model is the best at handling tasks like this. So let's go ahead and do a quick look at this agent and all of the rules it has to follow because it's my favorite agent I've built of all time. And then once we understand the agent, we'll go ahead and start kicking off the test. So let's go ahead and dive in. All right. So when it comes to this first test, which I'm calling Instruction Overload, we actually have a crew running under the hood, which has three different agents. We have a synthesizer agent, a newsletter writer agent, and a newsletter editor. So those are the core agents inside of this task. And I'm going to go ahead and show you all the instructions that we need to follow in order to go off and write a great newsletter that feels like a human wrote it and not an LLM. All right. So as you can see, we have three steps and three agents. So the first one, its sole purpose is to go off and build a strategy and take a brain dump from me and to go off and turn that into a great outline and a subject. So what we've done, as you can see, we have a ton of instructions in here. So like, hey, agent, please go off and follow these steps. So process the brain dump, come up with a subject line, and then develop an outline. Then what we do is give it even additional instructions like, hey, here is the strategy for writing a great headline. It needs to follow these rules. Here's a bunch of examples. So as you can see, this is like a really instruction heavy way to go off and create this agent. Then we go ahead and give it an example input and example output so that it can mimic what we want it to do in the real life. And then from there, once it has the subject line and the outline, we spin it over to a newsletter writer. This newsletter writer is responsible for taking in that subject line and outline and actually turning it into an actual newsletter. So you can see, once again, a bunch of steps and instructions on, hey, please do this for the introduction. Here's a deep dive section. And basically just, here's what you need to do in order to go off and basically create a newsletter that follows a specific template. And once again, we give it examples and example output so that it knows exactly what it needs to do going forth. So you can say like, hey, we're going to address our AI developers and follow the exact structure above, which focuses on following this template right here. And then finally, we have another agent who's responsible for going off and just quickly doing a one-time loop through this newsletter that was just generated to make sure it's following all the rules that we want it to. So review the introduction, make sure it follows all these other writing rules that I would do if I was to sit down and write a newsletter. So it's pretty cool. This newsletter normally would have taken me a few hours to write, but now it takes a few seconds. It's unreal. This is by far my favorite agent of all time. And you can actually go ahead and download this entire code completely for free. Once again, link down the description below, but enough talking about this agent. What I want to do now is go ahead and actually show you some of the results when we went off and ran this crew with multiple agents to see how it performs when we actually put it up to the test on instruction overload. So let's go ahead and quickly look at the final results and then we'll compare newsletters at the very end. So let's dive in and look at the test results for test number one, where we're trying to figure out if these agents can officially follow model instructions properly. So before we just dive into the results, I do just want to mention that all of the agents were able to successfully build a newsletter and in the next section, we'll actually look at the results. So you can see like, oh yeah, like this model did do a better job, but they did technically all pass. They all did generate a newsletter. Now what I want to do before just like diving into exactly comparing the models, I just want to show you what a run from one of these models looks like so you can understand what's going on behind the scenes. So as you saw in the code deep dive, what's going on is right out the gate. We had one agent whose sole job was to go off and, you know, take the brain dump and write an outline from there. It was supposed to go off and do a bit of writing. Once it was done writing, it was supposed to do some editing. And then finally we had a little tool call that was actually there to confirm that the word counter tool to make sure that we hit the word limit, which was close to a, we wanted a thousand words inside of these newsletters. And then finally, once it was done verifying that the final editor should just go ahead and tell us what the final newsletter is. So that's what a good run looks like. So now that you understand what's going on, let's go ahead and look at the results from these different agents to see exactly how they stacked up against one another. So right out the gate, the deep seek models took forever. That's the biggest takeaway. Deep seek v3 specifically really struggled. So it took over 10 minutes, which is crazy how long it took. You can also see that it took a decent amount of input tokens and output tokens. So it was insanely cheap to use, but it just took forever. So I would not be using this as an actual agent. Deep seek R1 on the other hand, still took almost 10 minutes to run, but it was very efficient and it did do pretty good results. You'll see here in a second, but okay. Results. I'll give it that at best. All right, next let's talk about Claude Sonnet. This one very unimpressed with it took long time, you know, almost four minutes, but it took an insane amount of input tokens. What this tells me whenever I see this number is as you can see over here, how we were constantly hopping from task to task to task. Like what this tells me is this agent is making multiple recalls over and over and over again. So instead of it being like one, two, three, four, it's like one, two, and it does it like six more times. We can actually look it up together so you guys can see the final results. So we'll look up the stack trace together so you can see, oops, sorry. Too many windows. So you can see for Claude Sonnet, this one struggled and you can see this is exactly what's happening. It's gross. So it's like, it's riding a little bit. No, is that it? No, I, it's not the right word count. Did I do it again? Nope, I didn't do it. So it's just over and over and over really struggling with word count and can not write a news article of the proper length. So that's why this one failed really hard. So let's go ahead and hop back to our results. So this one is a huge fail in my opinion, because it costs an insane amount of money. It took a very long time and it took an insane amount of tokens. So very unimpressed with this model. Next we're going to hop over to the OpenAI families, which actually did a really good job. GPT 4.0 did a really good job of just running through. It took less than two minutes to generate a really good result. It was a little pricey, nine cent for a newsletter. And then on the other hand, the O3 mini model absolutely crushed it. It was super fast. It took, you know, barely any tokens to get the desired result. The only other model that beat it was R1. And then finally, you know, it just generated the proper result. The other model that I was still really impressed with was Gemini Flash 2.0. This model ran pretty fast, a little over a minute. It did have a ton of tokens though, like, you know, so you, it's kind of repeating that same format that Claude was doing where it's like, do work. Nope, that's not it. Do work. Nope, not doing it. So it did that on multiple cycles, but it was really fast and it got to the desired result and it took like, it was one of the cheapest, it was the cheapest model. So it was fast and very cheap. So I'm going to give that one a runner up. So all on paper, the OpenAI families crushed it the most. And then the runner up was Gemini and DeepSeek was left in the dust. So let's go ahead and actually look at the newsletters to see what the actual final output was so we can look at the quality of the newsletters and not just the raw stats. All right, guys. So I'm super excited to show you what these agents were able to do when it goes to building a newsletter that followed a ton of super specific instructions. And just for a little bit more background, what I did is the input for this. I was like, Hey, please write a newsletter that goes off and talks about how you can use the four different types of luck to go off and build your own AI personal brand to attract opportunities to you. So that's what I wanted to write about. And here's the best one by far. So it specifically called out the right developer. So this is who it's supposed to be addressing. And then from there, what I did is I had four types of luck and I wanted it to make sure it included a number for each different type. So type one, it should have a number, and then it should kind of basically talk about the concept, give a few key insights. It needs to do bullet points. And then from there, it needs to just kind of have some takeaway lessons or talking about actual steps. And it kind of did that for everything. It's very easy to read. It's very skimmable, bullet points, numbering. So we basically want to make a newsletter that's super, super easy to read. Then at the very end, I wanted to kind of say, hey, let's do a quick wrap up. So it did that. It's also supposed to, you know, talk about actionable takeaways. So I basically put everything that we talked about into a nice synthesis. And then finally, what I did is I was like, all right, at the very end, you need to have a nice PS that kind of calls out to some of like other products and services. And you know, and it did it. So that's all around. It absolutely crushed it. So what I want to do next is show you the runner up, in my opinion, which was Gemini Flash. So it did a really good job too. It called out the right audience. So developers, then what it did from there, it kind of followed the same process. However, you can see this one, it didn't number everything. So not the best. And then also you can see that this one didn't use a lot of bullet points like it was supposed to. But the important thing is like it did stay pretty much on topic. The only thing is it did add in some extra stuff that like it wasn't supposed to, like it kind of went off the rails of what I said it should talk about, but it still stayed pretty much on track with following everything that I asked it to. The only thing I didn't like is when it came to like adding in tools, I didn't really talk about these tools. It was supposed to just stick more to actionable insights. So it did follow it, but not the best. And then, but all around it did do the right PS, but it did add in this random call to action that I did not ask it to do. So let's go and look at DeepSeek next and see how it compared. So DeepSeek absolutely crushed it, in my opinion, when it came to like the formatting. However, it did not follow what we were actually the core topic. So you can see engineer your luck three ways. That's not what it's supposed to be. There's actually four types of luck and it went ahead and just absolutely skipped one of them. So it didn't even talk about blind luck. So it just kind of like didn't understand there's four types. So this should be number one. And then, then we go from there. So maybe it got confused because like blind luck's out of our control. I don't know. But I was not super impressed with that, but it's still like, I do like the way it writes. It did follow my writing instructions really well. It made it very readable. So it did follow what I needed it to do, like the style and the format and the instructions. It kind of just needed some additional feedback and that's a common theme. If you want to check out my last video where I compared all these different models for just regular development practices, you can see that DeepSeek R1 is insanely smart, follows instructions to the T, but sometimes it just needs a little bit more guidance along the way and a few reminders to get to the desired result. So all around, it absolutely did great, but it kind of just, you know, got a little bit, it went too hard in some areas and it didn't do it enough in other areas. And I do want to just show you a few other ones. I think you might find pretty interesting. 4.0 has been my go-to model for a super long time. And once again, I just want to show you how it performed. So it called out the wrong audience, but hey, that is what it is. It did not add a subject line. So this was why this one failed. It didn't number the different types of luck. And then finally at the very end, it did do a pretty good job calling out stuff. Yeah. And then the only thing is like the curated resources, basically just, it didn't really actually curate resources. This is, should be actionable insights. So you know, all around, not the best. And then just to give you guys, to show you after we spent 30 cent, let me show you what Claude was able to do. So Claude, it should have said that this was the subject line, but at least it's there. So it did better than 4.0. Now it caught out the right audience. Then when it comes to actually like numbering stuff, it did do pretty good. And then all around, it did like kind of follow what it was supposed to do, but it kind of went off the rails. Like I never said to talk about like, you know, giving, I never told it to go this deep into like coming up with actionable steps. So all around it just picked that it was going to become a professional at giving you advice on how you can, you know, implement this solution. So this is more of like an educational newsletter and it kind of took it into like, no, this is a full blown strategy that you have to follow. So all around it finally got there. Now here's where it really failed. At the very end, it just went off on its own. It was like, oh, by the way, you know, you should get this and this and this. And it kind of just came up with this huge extra thing at the end, which has never been mentioned anywhere. So Claude, it cost a ton and it didn't follow the results. So as you can see, 03mini did exactly what we wanted it to. There was no really need for additional changes. So I would pretty much post this and I'd be very happy with the final result. So, but enough of that, if you have any feedback too on any of these tests or if you have other tests that you like need to run, let me know. But let's go ahead and start hopping over to the next test where we're going to be putting all these agents to the test against what I'm calling tool hell. So let's go ahead and hop over to test number two. All right, guys. So now it's time to dive into test number two, which I'm calling tool hell. Now, tool calling is one of the most important features of agents, and it's actually the main thing that separates agents from regular LLMs. A regular LLM just goes off and pretty much just generates a response. But an agent has the ability to take multiple attempts at the same solution. It can call tools to get additional information at specific times. So it's one of the most important features of making a great agent. So here's how we're going to test it. I went ahead and gave all of these models a bunch of different tools that it could possibly call. And I want to make sure that it can call these tools with the right parameters and in the right order. And to test it, what we've done is we've given our agent, I believe like five to six different tools where we've given it instructions on which order it should call the different tools and how it should pass the input from one tool to the next to really go off and tool hell. This is a very kind of intensive test. And at the very end, it's going to use all the tool results to generate a final result. So I'm super excited to show you the test I built for it. And at the end, what we're going to do is same thing that we did last time. We're going to go off and evaluate how the different models performed, you know, how much time, how many tokens, price, and did it pass or fail? And then we'll go look at the final, in this case, we're generating a poem and we'll see if the agents were able to successfully generate a poem and which one did the best. So let's go ahead and dive in. All right. So here's exactly how we're setting up this test to see if these agents can handle tool hell. So what we're doing is we have an agent that has five different tools. So you can see we're going to try to do everything from getting the current date, getting the weather, getting the stock price of Nvidia, search the latest news on Elon Musk. Then what we're going to do to try and trick it up is we're going to say, based on whatever weather you get from up here, I want you to use it to generate the square root. And then once you have everything, please go off and write a song using all this information and the theme of twinkle, twinkle little star. So that's exactly what we're going to be doing in here. So let's go ahead and look at the results to see how these different agents stacked up under the tool hell test. So here are the results for test number two, when trying to figure out which of these agents perform best on tool calling. So right out the gate, this one separated the pack pretty clearly. So as you can see, deep seek V3, deep seek R1 and Gemini flash all failed, but they failed for different reasons. Deep seek V3 just could never get this working. It's tool calling abilities are utter garbage. So it not use this for any tool calling agents. Next there's deep seek R1, which was, it could generate all the way through and do tool calling, but it would only work 20% of the time. So 20% of the time it would just generate perfectly valid code. But the other 80% of the time, just because it natively doesn't support tool calling and we're having, in Cray, we had to use some like extra features to get tool calling to work would just kind of not do what it needed to do and follow the proper tool calling process. So I'm going to give this one a fail just because it's not consistent and reliable. The next one is Gemini 2.0 flash. So this one failed for a different reason. This one was able to do tool calling properly all along the way. But then at the final step, when it was supposed to actually go off and generate a response, the model itself just didn't understand how it was supposed to go off and properly spit out the final result that you'll see here in just a minute. So it, like I said, it did do tool calling properly check, but the final output wasn't what it was supposed to be. So it's kind of like a pass and fail. Definitely worth, you know, potentially using for light tool calling, just like single shot tool calling and maybe with some better prompt engineering, it could have worked. But I'm going to give this one like a pass fail just because it did call all tools properly, but interpreting and using all those tools in the output, it failed. So maybe that's what I mean. Maybe the test wasn't a hundred percent, but the other models with the exact same prompts absolutely crushed it. So when it comes to all the open AI models and Claude, they did it with no, no concerns. And let me show you what a good run would look like just around the same page. So this is looking at, I believe, four Oh, and what you can see that we were doing is passing in the initial prompt of like, Hey, here's your job. Here's all the tools you have. And you can see it followed the desire structure. So it would go prompt, call a tool, look at the response, call another tool, get a response, call the next tool, response tool, response tool. And then here's your final output where it was go off and generate the final answer of twinkle, twinkle little star. So as you can see, this is what a good run looks like. And this is exactly what the open AI and Claude models were able to do. And I do just want to mention, I was a little bit shocked on these results, GPT four Oh absolutely crushed it. It worked super fast, faster than Oh three model, which is kind of like a shock. I was not expecting that. So yeah, this one definitely looked to be the winner on paper. So now let's go ahead and start looking at the results of the actual, you know, the final outputs from these different models to see which agent did the best. All right guys. So let's go through all the different results so you can see exactly what these different agents were able to produce so you can see which one worked the best. So out the gate, we agreed that the Gemini flash model was able to call all the tools properly and for the cheapest amount. But when you actually look at the final output, what you can see is that it called most of the information properly. So it got the date, it got the stock price, it looked up Elon Musk. But when it comes and it got the weather, but when it looks at going off and you know, getting the total amount of like, you know, we were supposed to get the square result of the weather, it never did that. So all around it never went off and actually, even though it called the tool and got the right result, it didn't spit it out. So this is just kind of like one of the things that this model, it did do the tools in the right order, but because it's reasoning ability was not the best, it was not able to properly do the tool call. So not a big fan of that one. All right. Now let's go look at GPT 4.0, which was like one of the second best performing models on here. And you can see that this one did awesome. It went off, it got the current date. It went off and did the weather. It went off. It looked up Nvidia's price. It went off, got Elon's information, and then it went off and squared it. So it absolutely crushed it. Same for the other O1 model. It was a little bit more, it actually used numbers and this one also crushed it. And then as you saw earlier, the DeepSeek models were not able to do things properly and the Claude model also, you know, it ran, but it just cost a good amount of money. So they all, all the models worked. It's just that the Gemini flash in its final output did not properly get the right information. So it's kind of like a half, it did it, but it didn't finally do it. So when it comes to actual results, it looks like the O1 models, sorry, the OpenAI models were the best. And, you know, once again, I'm going to say O3 Mini won this one, but you could also say GPT 401 won this as well, just because they did do the best and included everything and at a reasonable price and speed. All right. So now that we got that out of the way, let's go ahead and hop over to test number three, where we're going to start looking at RAG. So the ability for these models to go off and handle working with a ton of data, finding the right information, and then going off and giving the user the information they want. So let's go ahead and top into test number three. All right, guys. So welcome to test number three, where we're going to be doing a needle in a haystack test on these different agents. So basically what we want to do ultimately in this test is to make sure that these models can properly identify information in a vast amount of raw information and generate a response to it. So to test it, what we're going to do is we're going to give these agents a large amount of raw random data, and we're going to hide one key piece of information in the information it has to sift through. And then we're going to ask the agent a specific question, and it's up to that agent to hopefully generate the proper response and find that one key piece of information. And per usual, we're going to evaluate it, look at the scores, and then what we're going to do in this one, because it's just generating a single response, it's pretty much a pass-fail, and we don't have to look at the raw results. So let's go ahead and hop into test three so I can explain this a little bit more in detail when we look at the code. All right, guys. So let's go ahead and look at the code that's going to go off and perform this needle in a haystack overview. So we're going to say, all right, agent, your job is you're a research analyst, and it's your job to go off and answer queries that the user gives you. And you're going to go through and sift through a bunch of information and give me the desired result. And the specific job that it has to do, the specific task, is you have to say, what's the favorite thing Brandon Hancock likes to do? So that's the main question you're going to go off and try and answer. Now, like I said, what we're trying to do ultimately is we're going to throw in a ton of information into this agent for it to have to go sift through hundreds of thousands of tokens in some cases. And the easiest way we're going to do this is we have a get data tool. And all this get data tool is it grabs some information from a local file. We're not going to actually do a rag request, so we're not going to like embed things and then go off and then look through a vector store because, you know, getting a little nerdy about this, when you're doing an actual rag request, what you're doing is you're going through, oh, okay, there are, you know, hundreds of megabytes worth of data. And I just want to find the most, you know, three, five, 10 relevant pieces of information and pass it over to my agent. So that's, you know, that's more on the vector database embed search kind of side of things. And we're not trying to do that. We're just trying to swap this agent with a ton of raw data and make it to go through and look through that raw data to find the raw information. So that's why we're not doing rag requests. We're just saying, hey, here's a ton of data. Go off and look. Now, let me show you what I mean by a ton of data. So when we're testing the O3 mini model, it has a 1 million context, sorry, when we're testing the Gemini flash, it has a 1 million context window. So we're going to push it to the limit by passing in a context of basically passing in a file that has 900,000 tokens worth of context, which is wild that it can still look through this. And basically what it's trying to look up is it's trying to find this raw string right here, which says Brandon Hancock makes YouTube videos for about AI for developers and entrepreneurs. His favorite thing is when viewers just like you like and subscribe to the channel. So we're going to go ahead and start testing each one of these agents to see if they can sift through all of these different files to find the relevant information. And it is important to note that each one of these models has different input limits when it comes to context windows. So I made a bunch of different files that would work. And it's important to note a lot of the models, and we'll see this in a second, but a lot of the models actually don't live up to the total amount of tokens that you can pass in. It was breaking. So we'll talk to that more when we look at the results. So now let's go ahead and look to see how these models performed when we did this needle in a haystack test to see which models live up to the expectation and are able to successfully go off and research what the heck does Brandon like when it comes to his favorite thing. So let's go ahead and look at the results. So here are the results for test number three when it comes to looking to see how these agents perform in the needle in a haystack test. So let's go ahead and start looking at some of the worst models such as DeepSeek v3. So right out the gate, what you'll notice is this model and Claude Sonnet both failed. And as you can see, they were not even able to use half of their entire context window whenever we were passing in a large amount of data. So let me show you kind of why it was failing behind the scenes when we look at the stack trace. So what you can see is we were calling this research agent saying like, Hey, please go off and research. We were only at this point using 400 tokens. We called the tool properly. So pass on the tool call. Then we were able to get a response from the model. So you can see we fed in the additional 30,000 tokens into this, you know, v3 chat model, but because it was not able to answer in a single shot. So like, here's the result. What it had to do was then recall all that information. So we got 30,000 tokens in, then it's like, Oh, I don't know the answer. Let me call another iteration and pass that 30,000 in to a second call. So we got 30,000 plus another 30,000, which put us at the limit of 60,000 and it was able to pass. But the fact that it could not single shot the answer mean we are not using tokens efficiently. So all in all, you can see we used 90,000 tokens, but all we were able to pass in was 30,000 just because it was having to recall the agent over and over and over to try and get the answer. So huge fail, not efficient use of tokens because it's not smart. So that was a fail. Same for CloudSonic. I only could put in 26,000 tokens and it kept doing the same thing where it kept multiple times would call, get the answer, but it not formally find the results. So it kept searching over and over and over again and exploding and not basically hitting that eventually kept recursively calling and exceeding the 200,000 tokens. So I could only get it to work at 26,000 tokens input. Otherwise it would just fail. So huge fail on both those. So let's go ahead and look at the models that did pass and see what they did differently. So you can see, let's go ahead and look in this case, look over at GPT 4.0. And for this model, what we were able to do is go off and do what we were supposed to do, which is a single shot response. So, Hey, go look up stuff, do a tool call. Then we have a ton of information. It's supposed to write it in there. Give us the answer. And it did. So you can see in this case, we were able to pass in 125,000 tokens and it got the answer. So huge success for GPT 4.0. It actually executed the fastest out of all of them, which is a little bit surprising. I would have expected Gemini to do it, but it didn't. Maybe that has to do with the token throughput. So let's go and actually look at O3 mini next. So O3 mini has a much bigger context window of 200,000 tokens, and we were able to pass in 190,000 tokens, which is crazy. So we were able to pass in a lot more tokens, 70,000 more tokens, and it only took two seconds more. So that's wild how much more information it can process and still single shot the answer. Now let's also look at another model like Gemini 2.0 flash, which is by far the most like mind boggling one because it ended up being able to process over 900,000 tokens. And in order, whenever it processed 900,000 tokens, all it cost us was eight cents to do the entire search through all of it, which is crazy how cheap it is compared to the other models. So much information, so little money, and it was able to single shot the response. So if I was you and going off and building some rag applications that were going to be really data dependent and heavy, Gemini 2.0 flash knocks us one out of the park. It's not even close. So yeah, simple rag request, simple Q and A, chat bots, Gemini 2.0 flash is your new go to model. But if you want to do some rag and some other stuff, definitely these other models like O3 mini would probably be my other suggestion with how cheap and how much information it can process. So yeah, I hope you guys found this super helpful and hopefully it was pretty cool to see how all these models stacked up side by side when it comes to all these different tests that we've put them through today. All right, guys. So after all of that testing, which model wins? Well, hopefully it's pretty clear O3 mini is the best model for your agents. It's the most well rounded between its throughput price and everything else. It's reasoning just all around. It's a great model. Now I do want to give a few runner ups just to point out a few caveats. If you're building a chat bot, or if your project involves doing a ton of queries, Gemini flash 2.0 is the best model for these types of use cases. So I would definitely recommend using this because it's insanely cheap, insanely cheap to use, and it's 1 million context window is crazy. And just let you know, there are some bigger models out there. There's the Gemini flash 2.0, which has a 2 million context window. Definitely recommend checking that one out even more if you're going deep into rag land. And I do just want to mention if you are working on building a project and you really don't care about costs, you need to be hyper efficient on money. DeepSeek was a really smart model and insanely cheap to use, but the sacrifice was time. So definitely if you're looking for a process based agent, that's good at following instructions, don't care how long it takes. DeepSeek R1 was a great catch up. But the catch on that one is if you need to use tools, it's a no go. So strictly process based task, DeepSeek R1 does make a viable solution. But yeah, that's a wrap for this video guys. I hope you all enjoyed seeing all these models getting compared. Let me know if there's any other models you do want me to test in this agent setup. Love to try them out. And if you have any questions about anything today, please let me know. Oh, and real quick before you go, I want to do a few quick reminders. First, all the source code in this video was completely for free. So be sure to check out that link down the description below. You also will get access to that agent. That's my favorite one I've ever built where you get to create a newsletter in seconds and you'll get access to the other tests and test results that I shared in this video. Also, I have a free school community with over 4,000 AI developers of all skill levels. So if you're looking to hang out with some like minded people, definitely recommend checking it out. We do free coaching calls every Tuesday night, and we'd love for you to be there to help you get unstuck on whatever you're working on so you can keep building awesome AI projects. But that's enough guys. I have a ton of other AI related content on this channel. Everything from Cray I link chain full stack AI tutorials. It's all here. So I definitely recommend checking out whichever video pops up next. But hope you guys had a great time watching today and I can't wait to see y'all in the next video. See ya.