Run a local AI with Ollama

Okay, you want to get started running AI locally on your machine because you're sick of all these AI wars and all these big corporations, right? I can help you get started. Hi, I'm Jeremy, I've been working in tech for 25 years and I help people. There's a lot of people out there talking about this but they're going to start off talking about all the technical stuff, harnesses and coding agents and wiring things together. You don't have to go down that road just to get started because a lot of these people forget that not everybody's using these to code. So I'm going to show you the very basics of getting AI running on your machine so you can play around, see if you like it and if you like it, continue with it. Plus I'm going to show you some extra tools that come with it. The first thing you're going to do is go to OLAMA's website, so just type O-L-L-A-M-A, it's the first result most likely. You're going to click on that and then from here I'm going to show you one thing first. So you're going to download the app but I want you to look over here where it says models. Don't worry about pricing, you're not going to pay for it. So here they have a large library of models which we'll talk about later but they have tons to choose from. You don't have to be stuck with whatever they hand you like ChatGPT or Claude or Gemini. You get to decide which model you're going to run and it can be a bit overwhelming, you don't know which model to use. You can use the recommended ones just to get started then you can research these later. So now, like I said, you're going to download this desktop app, you're going to install it. Pretty much works the same on Windows and Mac and once you install it, it's going to look like this. Very basic. So what you have is something very similar if you've used the ChatGPT desktop app or the Claude desktop app or even Gemini in the browser. It's all going to look very similar and you have a chatbot and what you have here is you get to pick which models. If this little download symbol is there, that means that you have to download it before using it. So I'm just going to use what is listed here is Gemma 4. You can pick whatever model you want to. Gemma is Google's model. They're one of their open weight models and you can pretty much interact with it exactly like you would ChatGPT or any other. And it thought about it for about three or four seconds just like it would with ChatGPT on a slow day and there's your response. Now keep in mind, if you're running these local models, you should have at least a decent graphics card. Here I'm using a Mac Studio which has a great graphics card. I have a pile of laptops back there that you can't really see. All of them have Intel graphics cards and about half of them run this pretty well and they're newer. I think the oldest is about four years old that has the better graphics card, the Intel graphics card, and it still runs pretty well. So just keep that in mind. Pay attention to your graphics card if you're going to be trying this out. But just like any other chatbot or any other agent as we call them or any other model, it has a lot of features built into it. So like let's say for example, write me a quick Python script. I'm just the very basics of the basics to show you that it actually has coding capabilities if that's something you're interested in eventually. It does have that ability baked into it already. It's trained on a lot. It's trained on a whole lot and depending on which model you use, a lot of them will be trained on a lot. Not as much as ChatGPT, not as much as Claude, not as much as the Frontier Google models, but they're pretty close actually. For most people, for what you do, it's very close. If you want to ask it questions, it's been trained on lots of things and it knows lots of things and it can answer lots of questions. But one thing I want to show you is this desktop app is pretty limited. It can't do a whole lot, but what they've done is they've included some options here. So let's say you are into code. You can actually launch Olama from the command line into whatever coding agent that you use. So if you use Claude code from the command line, it tells you how to do it. If you use ChatGPT codecs from the command line, it tells you how to do it. And then you have all these other harnesses here that you can play around with, but these are new. So I'm going to show you first, I'm going to show you ChatGPT. Once you enable that, it asks you to restart ChatGPT if it's already running. And then what's happened is I am now in the ChatGPT codecs app and I have a selection of models down here to choose from. So I can choose any of these models to run inside the ChatGPT codecs app. And if you saw a moment ago, you can actually adjust how intense the thinking is. So if you crank it all the way up, it's going to give you better results, but it's going to take a lot longer to give you a response. Right now I have it set on the fastest setting, but the same thing, like let's say, write me a Ruby script to get the weather, you know, something stupid. I'm just going to have it run a simple task to write a simple Ruby script to get the weather. And now it's done. And this is the code that it came up with. And that took seconds. Now mind you, I'm on a Mac Studio, so it's going to work pretty fast because of my graphics card. But it has pretty much all the same capabilities built into it. If you're the kind of person that lives in the chat space, this actually works pretty well. If you're the kind of person that does code, this also does pretty well. But that's the ChatGPT. Let's take a look at Claw. And pretty much a very, very similar thing. Write me a Lua script to list all files in current. And I'm giving them simple tasks because I don't want to take up a lot of time for the video. I'm just demonstrating that it can do these things. And now it's done. I can either open it with, well, that app's not going to work. I'll just show it in the folder. And I'm going to open it with VS Code. And sure enough, it gave me a simple Lua script that does exactly what I set it to do. It actually formatted it pretty well. Yeah. And it gave it nice comments and everything. So again, I'm not going to demonstrate all the features of this. But I did want to show you the very basics of it. And let's say one of the models you want to use is not listed here. What you do is you go back to the website. You go back to the models. And it actually tells you how to run it. So again, you want to pay attention to where it says cloud. If it says cloud, you have to pay for it. Because there is a paid version of Olama. And those ones with the cloud next to it, you have to be subscribers to use it. But there's all these other ones. And it tells you exactly how to run it. So let's say I want to run this one. What I would do is open up the command line and run exactly the command it told me to run. And what it's doing right now is it's downloading that model I told it to run. Because it has to download it to your computer first before it can run it. And this is going to take a while. This is a very large model. It's 104 gigabytes. I'm not going to let that download all the way. I'm just going to show you what it looks like when you run one. So let's say I want to run Lama, which is already downloaded. It takes a couple seconds to think about life. And then there it is. It's loaded. So that's how you do it at the command line. That's how you do it in the desktop. This is just the very, very basic for those who are just getting started. And they don't want to watch all these tutorials. I gave you a very quick overview of how to do it. This is how you get started. After you get comfortable with this, then you move on to the next steps, the more advanced steps beyond this of what it can do. But this is just the beginning. Go ahead and get Olama and get started today.