From the course: Hands-On Introduction: Data Engineering

Unlock the full course today

Join today to access over 24,700 courses taught by industry experts.

Solution: Solutions to ETL questions

Solution: Solutions to ETL questions

From the course: Hands-On Introduction: Data Engineering

Start my 1-month free trial Buy for my team

Solution: Solutions to ETL questions

“

(upbeat music) - [Instructor] Congratulations on completing your first challenge. In this video, we'll review the solution to our data engineering challenge. As always, let's start with our business request. It sounds like our user is doing analysis on S&P 500 companies. We know that this data must be pulled from somewhere. That is our extract step. I've highlighted it in blue here. Next, we know that our user wants to understand the aggregate number of companies in each sector. That's our transform step, and it's highlighted in orange here. Finally, we know that our user wants this output in a SQLite database. That's pretty straightforward and has a clear indication of the load step, that is highlighted in green here. Great work. We now know which aspects of the ET&L process correspond to which parts of the business request. Let's switch to Codespaces. First, we want to make sure we understand what the output of our ETL process will be. We know that our prompt requested that we write…

Contents