Different ways to create an AI using Python, to play the game Snake!

what's going on guys welcome back to the channel in today's video i'm going to show you three different ways to create an ai computer player for our game snake [Music] so if you didn't watch the previous tutorial on my channel um i showed you how to create a snake game in python using the pygame module and i will add a card to that tutorial video as well as a link to it in the description and a link to the github if you want to grab that code for this video i'm not going to show you how to build the entire snake game from scratch we did that already and in this one we're going to focus on how to create a computerized player before i begin thank you coding kasowery for the suggestion um for this video he said hey cool video how about coding different ais that can play the game perfectly when i posted the initial how to build a simple snake game video so if you have an idea for what you'd like to see in a future video or any concept that you'd like covered just let me know about in the comments below be sure to leave a like on the video subscribe to the channel helps me out a ton and stay tuned to maybe see your suggestion in a future video so when i dug into this a little bit and started creating a computerized player i realized there's actually quite a few different ways you could have an ai that plays the game snake and there's actually a really simple way to build an ai that's technically going to get a perfect score every time and it looks something like this this is basically a version of snake where the computer knows to just take the absolute longest route through the board so no matter where the food is it actually doesn't really care because it knows eventually it'll hit the food and it'll know no matter how long its body gets to be it'll be safely stored behind it until it gets the absolute maximum score this is a super simple solution it really only requires like three lines of code basically start the game with the snake going upwards and if you hit the top of the board turn right and then go downwards and if you hit the bottom of the board turn right and then go upwards and it'll do that the whole time if you have a version of snake like we do where you can pass through walls which is the traditional style of snake then this is all you have to do to get a perfect score but this is one it's pretty slow it looks basically nothing like how a player would play the game so it's not exactly optimal um and it's just it's kind of slow and it's kind of boring and it doesn't really require a lot of coding knowledge so we're not going to cover that anymore that is the most basic way to get a snake ai that's going to get a perfect score every time and then the second kind of ai that this video is not going to cover um is what would be considered like maybe more true ai which would be like using a free agent where you don't give it any rules you would want to use the pie torch module for this and essentially do reinforcement learning or you basically introduce a little bot that has no idea about the world that it's in or anything it just knows that high score is good lower scores are bad and you let it do thousands of iterations and learn what it has to do to get higher and higher scores that's like machine learning level ai i'm gonna keep this a little more introductory but if you are more interested in that style of ai and tutorial there's a series by a youtube channel called python engineer that i recommend you go and check out he does a great job not only showing you how to create uh that sort of ai but also explains a lot of the theory behind reinforcement learning so uh i don't know him he's not sponsoring the video or anything he's just it's a good series so um check that out if you want to as well but what we're gonna do in this video is show how to actually program a snake ai that's going to act basically like a player and make decisions like the player would and pursue food similarly to how a player would pursue food so um without any further ado let's go ahead and get into the tutorial all right so we're starting here with the python pie game uh version of snake that we built in the last video on the channel again there will be a link to the github for this code as well as the uh tutorial itself on the channel in the description below so if i don't have a card on the video right here then just check out the description if you want this code but this is what we're going to be starting from as we create an ai so let's go ahead and get into it so basically any version of ai we're going to want to take our snake that we created in the previous game and basically get rid of its ability to uh to control our movement or give it the ability to control our movement get rid of the player's ability to control the movement so i'm just gonna repurpose this handle keys function because we already created a spot that was supposed to control the snake's movement we just called snake dot handle keys every iteration of our loop so we'll go ahead and remove the spot where the uh where the player can tell the snake how to turn and we're going to start creating some logic for the snake playing on its own so first thing we want to do i'm just going to get the snake's head position right at the beginning which again if you aren't familiar with the different functions we put in this snake class you may want to consider checking out the first video or just follow along and understand there's going to be some code written already that maybe you can pick up as we go along or you're going to want the explanation so we're going to get the head position which is going to give us where the head of the snake is as an x and y coordinate and we're going to want to compare that to the foods position so what i think we should do is pass in when we do handle keys pass in the foods position which will be easy enough we just go down to where snake dot handle keys and we add food dot position so we already had uh the foods position as a xy coordinate from the previous tutorial we just want to pass this into the handle keys function now all right and then the basic first step of the ai is going to be make sure the snake understands where its head is relative to the food and pursue it so figure out how to get closer to it and to do that i propose we start by creating a tuple variable so a set of coordinates essentially a two-dimensional vector that's going to tell us how far from the food in the x and y direction we are and to do that i'm just going to do the x and y coordinates of the head minus the matching x and y coordinates of the food and so basically you can look at this as if head let's say is further to the left than the food this will be a positive value and that'll tell us that we need to go right to find the food and if the head's y value is greater than the food position's y value that'll tell us that we need to go down but if these are negative then that means left or up depending on obviously left and right and so the simplest thing we can do would be just excuse me would be essentially this if distance 0 so this is the x coordinate is greater than zero self.turn and go left so this basically means we just checked the x coordinate and we are too far to the right the food is to our left and then we can do another if distance zero is less than zero then we want to turn right and you could also do these as l ifs i'll kind of explain why that's up to you in a second i'm just gonna copy these two real quick and we're gonna do basically the same thing for up and down but now we're checking if distance one so this is the y coordinate if that's positive or negative and if it's positive because of the way the pie game grid works positive actually means that head is going to be further down let me make sure i got that right further down because 0 0 is in the top left of your screen in pi game so larger y values are closer to the bottom of the screen so if it's positive we want to go up and if it is negative we want to go down okay and honestly just right here what we did you have the most basic version of a pi uh pi game python snake ai so let's go ahead and run that just to take a look at what that does now um what we'll see after it gets a few points here is uh it's going to pursue the food in the most direct way it possibly can um but at the risk of its own body so it it will at this point drive into its own body as it just did there to get food because we haven't given it any other conditions and told it that that's bad um so another thing i'm going to do for this tutorial down where we have clock.tick controlling how fast the game moves i'm going to step that up from 10 to 20. 20 should still be slow enough that we can view it to talk about it but now it's going to move much faster because the computer is playing it doesn't care about the speed uh so you can see right there it's it's pretty cool actually it makes decisions essentially like what a player would do which is locate the food and go after it but obviously when it turns too rapidly after it gets to around the 20s it destroys itself because it doesn't know that turning into its own body is bad so i had this running um on my version where i was also totalizing the total number of games and averaging the score and just at this level it gets to around 20 which is not difficult to do as an individual player and so our goal i think would be get a score at least around what a decent player could get so this is cool it's basic but let's go one step further which would be warning it that it cannot turn into its own body so basically it will only make a turn if uh it's if it's not going to hit itself so let's start there so what we want to do is come into the if statements for all of our distances oh before we move on i promised i'd say uh why these could be ifs or lifts um and basically it's up to you if you want to just understand if you're in a situation where it needs to go left and it needs to go up if you do it in this order then it's always going to turn left or right before it turns up or down so it's going to control which one gets precedent um if you do all of these ifs it still kind of happens that way because it's going to come top down that's how python code works um so like for example if you have to go left and you have to get up to get to a piece of food it's always going to be looking to check left before it checks up it doesn't make a significant difference i played around with the order of things um and and what they totaled uh for different versions of it doesn't make a significant difference to the score but if you ever are watching it and you notice it always tries to pursue left and right before it does up and down it's because of that okay so what we want to do for the next step of the ai is we want to add a check to make sure that it won't turn into itself and so this has to be a little bit customized for each turn so what we want to do is add another condition before we let them turn left and we want to include and head 0 minus and then grid size and then head 1 is not in self dot position so we already have this list of everywhere that the body is currently occupying and i know some people have mentioned they don't recommend putting like uh self.positions list they would just keep track of it on the board like have a board of all zeros or negative one and when the body is currently in that spot have an integer value that counts down that's positive and counts down how many more turns that spot's occupied you could do that too you would still have to check that the board was not occupied then however you do it but what we just said here is okay i think we want to turn left because the food is to the left but i just checked and one grid to the left of where the head is right now is actually occupied so this is saying both have to be true the left space has to be unoccupied and uh the food has to be to our left and so hopefully that makes sense but we're gonna copy that and do essentially that but modify it um for each turn now so let's say for the right it's going to be the same thing but it's going to be head plus grid size and the reason it's plus grid size is because these head 0 and head 1 are the x and y coordinate for each square so it's not just like 0 1 2 3 4 as you go across it's whatever square you're at times the grid size which in this case is like 20 because it's i think 480 wide and we have 24 squares so uh to check that the right square is not occupied it's just the same formula but plus grid size and then it's very close for up and down it's just obviously uh instead of checking that left and right are not occupied we're checking the up and down or not occupied so it's head one minus grid size and then the checking that it's open down is going to be head 1 plus grid size and i think i have one too many space in here alrighty so now this is not going to let you turn into your own body so it's not telling it what else to do but it is saying you're not allowed to turn into your own body and we can go and take a look at this the most common way we were dying before was not actually turning into our own body but it was not turning away when we hit our own body so like occasionally you will um take a turn that's not going to directly hit your own body but 20 paces down the line like it just happened there you don't turn out of the way so this is going to prevent say you had your entire body curled in two consecutive columns and i don't know if this randomly generated version is going to give us is going to give us the scenario that i'm talking about but if your entire body was folded in two consecutive rows it wouldn't turn into itself but it will turn in the direction of its body and eventually hit it so again i had just with the change that we made i let that one play out for quite a while oh you can see just there actually it did what we needed to do um i let that one play out as well for quite a few games and it only improved the score from an average of about 20 to an average of about 23. so that's obviously that's still not really ideal uh it's definitely getting better but this next change is going to actually get us up to about 35 40. so that's what i really want to accomplish for this video and the next chapter is going to be turning out of the way when you sense that the spot you're moving into is occupied by your body so this is a really important one and it's also a fair bit more uh coding intensive than the last two that we did so let's go ahead and put it together so what i want to do in our handle keys function and this is significant enough code that you could definitely break it out into its own routine but i want to create a new loop every time we check this to see how big of a um if we are about to collide in the left or right directions so to do that i want to say i'm going to create a new variable that we'll just use locally so you don't have to do self.colliding you can just say colliding but what i want to check is if self.direction is equal to down so this means we're moving downward and and now similarly to what we did down below to check that it wasn't occupied before we turned we're going to check that head 0 and then head 1 plus grid size so it's literally just like the check we do before allowing the player to turn down we want to check that that square is not occupied so what we'll say is if we're heading down and the next square is occupied so in self dot positions then we're going to take this colliding variable and set it equal to down and then we're going to copy this exact code and do it four times here so if we're heading upwards and the square above us above our head is occupied then we're going to say that we'll be colliding upward and we'll use this to turn left or right in just a second so then we want to do the same for left and right so we'll say left and then we'll say right and then we're going to make the colliding instruction left and then we'll say colliding right and we're gonna get these grid sizes so if we're heading right and the next spot to the right is occupied then we'll say we're colliding the right and if the next spot to the left then we'll say colliding left and i put too many spaces there okay so this should tell us if we're colliding left right up or down in the direction that we're moving but what we need to do now is make the decision to turn out of it um when we get that information and so to do that we're going to go down to where we were already controlling whether to turn left or right and i'll show you the simplest way to incorporate this but then i'll also show you how to make it better as we've been doing this whole video so the way to incorporate this step now into our decision making matrix is let's say that we're colliding up or down then we don't actually really care too much we want to make sure still that this is true so we're not turning into ourselves but now we're going to add an or statement to this guy so this guy is saying that well the main scenario where we'd want to be turning is if we're pursuing food but we're going to say or and then in here we're going to add some new conditions colliding is equal to up or colliding is equal to down and you're probably saying that doesn't make perfect sense because um well what about turning right and that's why i said we're going to get to just next so i'm going to go ahead and basically copy this exact statement for up and down and i'll just say that we're going to turn down for now turn down for what um control v so right now if we sense collision left or right we're going to just turn up and then if we sense collision up or down we'll turn to the left so this should be a basic way of avoiding our own body but it's not smart it's just a basic avoidance so let's go ahead and take a look at what that looks like and we'll just need to give it a second so it gets long enough to where it could hit its own body but i'll point it out when i see it hopefully around like the score of 20 ordinarily there's a turn where it would hit itself and in this scenario it should turn at least once out of the way so you can see there and there we get a few of these rapid little shifts but you saw if you saw why we just died there it's because it wasn't calculating whether it was a better decision to shift to the left or shift to the right just no matter what when it's about to hit itself it was turning left and it was turning up so the next step is making an intelligent decision about whether there's a larger gap to the right or to the left of it and so to do that we're going to come up above our collision detection this is still correct this is still good but we're going to add something new that i'm going to basically call gap detection and so i'm going to create these variables that are unnecessarily large so left gap right gap and all i'm doing is when i'm initializing them i'm giving them large values we'll say top gap and bottom gap and that'll be clear why in just a second here so what we're going to do is we're going to say for i in range and then uh we're going to say from 1 to integer of grid width so this is saying and you would want to do this separately for the x and y for the left and right gaps then the top and bottom gaps if you didn't have a square board we have a square board so i'm going to use this exact loop here for top bottom left and right but you'll see what i'm talking about in a second we're going to check that every square not the player's head which is why i'm not including zero here we're gonna check that every square on the board um whether or not it would uh be occupied right now and we're going to report those as the left and right gap so let's just go ahead and get started so i don't have to talk i can show you with code what we're going to do first is calculate the top gap so how much space we have above our head and we'll say if the head square that we're in so head 0 head 1 which is right where our head is minus i times grid size if that's in self.positions self.positions then we want to say that top gap is equal to i so let me just reiterate what's going on here real quick actually we want one more check if top gap is less than i because obviously if we find out that two spaces away we don't have a gap that but three spaces away we do have a gap we don't want to overwrite we want to find the closest position to the head um so let me kind of boot this up real quick so i can talk about what we're doing so essentially looking at this grid uh we want to iterate through the entire map and check for squares above the head to the left of the head to the right of the head and below the head and say how large of a gap you have before you hit a space that has your body in it which is going to tell you how far you are from collision on that side so what we're doing here is we're checking the square one above the head two above the head three above the head and as soon as we find one that's equal to an occupied space we can stop calculating there which is the purpose of the if top gap is less than i top gap is equal to i and we're going to go ahead and copy that if statement for all of these calculations and we're going to put it in here four times all right and so we'll say uh for calculating the bottom gap it's going to be the same thing but it's going to be plus i times grid size and then if bottom gap is less than i and that's why i give i uh a hundred to start because we're trying to calculate uh how big of a gap we have to the top and to the bottom and then we're going to do the same to the x and the y and i don't think i finished that thought but the reason i start all of them at 100 is because as soon as we find one that's like two squares away three squares away that'll always overwrite a hundred if i were to put some dummy value in here zero or i could even say like the number of squares which is 24 which is technically the the farthest away anything could ever be there's just there's no reason to not make these huge values initially so that once we hit something for sure it'll get overwritten okay so now we're checking if there's a gap to the left and it's obviously pretty similar code for all four directions and then here we'll be checking if there's a gap to the right okay and now this is pretty easy to incorporate into our decision making because what we want to do now is say okay we're moving downward or i think we want to turn left because the food is in the direction or we're colliding upwards or downwards but now we want to add a condition to this colliding bit here so or colliding up or colliding down and and pay special attention to the parentheses that we have here so we're saying that if we're about to collide in the up or down direction and the left gap is greater than the right gap so we're saying we have more space if we go left than we would if we went right in that case then we want to um do the same thing as we would have uh before so now we get to take this or statement and we can put the exact same or statement inside of our right turn logic but this time we're just going to flip it to say okay we're about to collide up or down and um right gap is greater than left gap and you want to pay special attention this it doesn't really matter which one you pick but say they're equivalent say you've checked left and right and there's no collision then uh you're going to want to make one of these greater than or equal to it doesn't actually matter a ton which one technically speaking you could check whether you're further to the left or right of the board and make a decision there but this is the simplest way to just make sure that if you're about to hit and the two gaps are equivalent you still do one of them because you always want to turn out of the way okay so now let's go ahead and do that to our up and down turns so we're going to say or colliding and then add an and statement in here and this is to turn up so we're going to say and top gap is greater than bottom gap and i'll put the greater than or equals to first uh two two two two or and then we're gonna copy this whole thing and if distance is two put it right before the and and drop the or right in there so if we're colliding in the left or right direction and the bottom gap is greater than remove the equal sign greater than the top gap now we're going to turn down so to really quickly review everything we're checking before we turn down we're saying okay the food is somewhere below us on the board or we're about to hit our own body in the left or right direction and we've calculated that there's more space below us than above us and we've calculated that the spot we're about to turn into is not in our self.positions so this is pretty smart now pretty smart ai let's go ahead and run it enough talking about it and what you'll see especially if you have this and you're following along on your own this now is smart enough to average in the 30s i don't know if we'll get lucky on our first game here and get a score in the 30s um but i let this one play and i mean really play i left it on like uh through dinner just to um come back and see what kind of scores we had and through 568 games i was getting an average score of 36 with this spot and that's crazy actually i think we got exactly 36 on our first playthrough um but this is actually finally pretty satisfying to watch it's making some intelligent decisions and quick movements sort of similar to what a good player would do it's turning to get out of its way it's making an intelligent decision about which way to turn so that it gives itself more space the next thing to do would be figure out how to calculate that gap and include the boundaries so make sure that you get out of the way after that it would be adding some code to make sure that you don't turn into a trap so like if your body is creating a horseshoe you don't want to turn into that horseshoe even if technically it satisfies all the other conditions those are what i would recommend for the next two steps to get you to a really intelligent snake game this already is pretty solid um ai and it's far more enjoyable to watch than just creating a an ai that's going to map through it and i think it requires a little more coding know-how and actual like critical thinking than just importing a module that's going to put a bot put an agent into your code but you can tell just from watching it and playing it on your own this is not a completed project but i think with what we've talked about and what you've learned in this tutorial this would be a really good spot for me to stop suggest you take it the extra step and let me know in the comments if you got further if you found an improvement or if you really would like to see more if you'd like to see me take this project the next couple steps let me know about in the comments below if there's enough interest then i definitely can make a follow-up to this video where i show you how you would make the kind of things where you'd avoid turning into a dead end you would be able to make decisions that would turn to the left and right and use the boundaries because obviously right now you can see the snake is never going to voluntarily turn to go through the left or right or up or down boundaries because we haven't told it how to calculate that that's actually the fastest route so i hope you found this enjoyable again i really appreciate likes on the video subscribes to the channel kind comments uh great feedback all that stuff helps me out a ton i really enjoy doing these youtube videos and the support makes it possible so thank you so much be sure to let me know if there's anything else you'd like to see in the future or if you have any comments as always good luck with your projects and thanks for watching thanks bye