Database vs Data Warehouse vs Data Lake | What is the Difference?

What's going on everybody? Welcome back to another video. Today we're going to be taking a look at the differences between a database, a data warehouse, and a data lake. Now when I was first starting out, I'd only ever heard of a database. I think that's what most people are familiar with, but I had never heard of a data warehouse or a data lake. And so in this video, we're going to be walking through the differences between each one of them, as well as how they kind of connect with one another. So let's jump onto my screen and get started. All right, so we're going to be taking a look at a database, a data warehouse, and a data lake, but let's start with a database. Now when someone says a database, typically they're referring to a relational database. Now a relational database can capture and store data via an OLTP process, which stands for online transactional process. So when a company completes a transaction and sells an item, it'll record that within a database. And that data has the ability to be live, real-time data. Data in a database is going to be stored in tables, which has columns and rows, and it's going to be highly detailed, which means you're going to be able to go in and see every single aspect of the data. And databases also have a really flexible schema, which means you can go in there and kind of change things as you go to make it work for what you need. Now a data warehouse is also a database, just like we were looking at before, but it's going to be used for analytical processing, or OLAP. OLAP stands for online analytical processing, and it's created to basically analyze huge amounts of data. Now if you notice on the last slide, there were these three databases, and they were just kind of sitting there, and they were storing the data. In this visualization that we have on the right, these three databases on the bottom are all aggregating and sending their data to this data warehouse via an ETL process, which is where it extracts the data, it transforms it, and loads it exactly how they need it in this data warehouse. And that's how data is put into the data warehouse. It isn't getting it directly from the source, but it's being put into a database, and via the ETL process, it's being updated as it goes, or whenever the ETL process runs. A data warehouse will always have the historical data, but it won't always have the current data, unless the ETL process is running every single day, or very frequently. The data in the data warehouse is also a little bit different, because we're doing this ETL process to get the data in there. We're not actually putting every single piece of data, or every column and row in there. We're typically summarizing it, and then putting it in there, which will allow us to process that data for our analytical purposes much faster. Now a data warehouse is going to have a much more rigid schema, so you really need to plan ahead with how you're going to put your data into a data warehouse. It's not as flexible as just a database. So now let's look at some of the key differences between a database and a data warehouse. A database is going to be used for recording transactions, or a data warehouse is going to be used for analytics and reporting. A database is going to have fresh and detailed data, where a data warehouse is going to have summarized data, and it's only going to be as fresh as the ETL process is created. A database is going to be a little bit slower for querying large amounts of data, and when you do query large amounts of data, it can actually slow down the processing of all those transactions. A data warehouse was designed for the exact opposite. It was designed to be very fast at querying and not slow down any processes, because it isn't part of that transaction processing at all. So now that we've looked at a database and a data warehouse, let's take a look at a data lake. A data lake was basically designed to capture any type of data that you could possibly want. It could be a video, a picture, an image, a document, a graph, anything you could imagine that you'd want to put in a database or store in some way, you can store it in a data lake. Now there are a ton of use cases for a data lake, but I think people who work with machine learning and AI get to use it or benefit from it the most. They can use all that structured and unstructured data and create models to really use it in its raw form, where if you want to use it for analytical purposes, typically you're going to have to clean it up a little bit and do a little bit more work to actually make it usable. And so a data lake is just that, it's this lake where you can basically throw any type of data in there, but it's not always super usable because you're just putting it in there in its raw form. If you want to use it for analytical purposes and reporting, most of the time you're going to want to clean that up and put it into a database or a data warehouse. So now when we're looking at all three, they are all different and they're all used for different purposes. So no one option is better than another for your data. If you're using it just to record transactions, a database is what you should do. And if you have a large amount of data, that's just too much for your database to handle, it sounds like you might need a data warehouse. And if you have all this data, they have no idea what to do with, or it's unstructured, it's semi-structured data, you can't fit into a database. Well then I highly recommend using a data lake. There really is no one size fits all. All three of these can be options for different uses. And in fact, you can use all three within one company for just different things that your company needs. So I hope that that was helpful learning the differences between a database, a data warehouse, and a data lake. Again, I had really never used a data warehouse or a data lake when I first got into analytics, but now that I've gotten hands-on experience with all of them, they're all really interesting to use for so many different things. So thank you so much for watching this video. I really appreciate it. If you liked this video, be sure to like and subscribe below, and I'll see you in the next video.