OpenTube

End to End Data Project with Microsoft Fabric - Data Engineering, Data Factory and Power BI

Pathfinder Analytics

157,853 views

🚀Join me in this tutorial as we build an end to end data engineering and reporting solution using Microsoft Fabric.

This tutorial uses the USGS Earthquake API to build an end to end data solution focusing on the Data Engineering, Data Factory and Power BI experiences.

⌚ Timestamps
00:00 Introduction to the Project and Solution Overview
03:20 Creating a Fabric Workspace and Data Lakehouse
06:48 Overview of the USGS Earthquake API
08:04 Using Python Requests to Fetch API data
14:40 Coding the Bronze Layer Processing Notebook
21:40 Coding the Silver Layer Processing Notebook
30:40 Coding the Gold Layer Processing Notebook
41:14 Updating the Default Semantic Model and Creating the Power BI Report
52:36 Using a Data Factory Pipeline to Orchestrate the Data Processing
01:03:10 Project Debrief

🔗 Links and Resources
GitHub Repository: https://github.com/malvik01/Earthquak...
Earthquake API: https://earthquake.usgs.gov/fdsnws/ev...
How to get a Fabric Free Trial without a work email: Start a Microsoft Fabric Trial without usi...
Unix Timestamp Convertor: https://www.unixtimestamp.com/
Reverse Geocoder Python Library: https://pypi.org/project/reverse_geoc...
What is the Medallion Architecture: https://learn.microsoft.com/en-us/azu...
Fabric Environments: https://learn.microsoft.com/en-us/fab...

📚 Check out my Udemy Course
Microsoft Fabric: The Ultimate Guide: https://www.udemy.com/course/microsof...

20 Comments

@hypnotheraps5 7 months ago

This was the best tutorial on this subject I have ever seen. Crystal clear explanation. You are a true professional

@Austinger 11 months ago

I'm new to this area, I really like this hands on meaningful way of learning instead of playing with abstract concept data, and particular like the way you break things down, save me loads of time to explore and understand the code. Great job mate.

@bill_manassas 1 year ago

I've watched three paid courses on fabric. This is the best course by far!

@poojasathe2502 4 months ago

Excellent hands-on tutorial! It definitely helped make Fabric less intimidating, plus I got to build my very first Lakehouse and Data Pipeline!

@mosesakwagiobe4438 9 days ago

I have never seen a tutorial on youtube so crystal and easy and enjoyable to watch let alone on Fabric. I landed here because i'm prepping for DP600. this has been helpful. Thank you. One question though, shouldn't the Country code and Signifcant class columns have been done in the silver layer? i thought the Gold layer is just a clean copy of Silver which is then expose to the analytics/power bi for viz? Should more manipulation be done on Gold? Thanks

@soumenchandra7633 11 months ago

Every starter should see this video, it's gem

@wezzuzondu8872 9 months ago

All hands on approach in this video which cuts out plenty time that would have been spent on abstract theories. Excellent work....Thank you!!

@lalller 1 month ago

thank you!! i loved the video so much u covered so many topics i could learn a lot

@wesker1413 8 months ago

Explained very very clear and step by step, I am looking forward your next Fabric project. Thank you.

@hettie2808 4 months ago

This is such a great resource, detailed and yet concise. Thank you

@ahmedchelly1253 3 months ago

You deserve so many more subscribers and views. Your voice is amazing, and you explain everything clearly and get straight to the point. I know how much work goes on behind the scenes to make videos this good. I hope you'll create courses on DP-800 and DP-700 in the future!

@faisalhafiz8371 1 year ago

Clear, simple, and easy to follow. The technical concepts were well explained, and I had no trouble following along with the video. Excellent work — thanks, mate!

@bezsmertna_analityka 1 year ago

I am very grateful for such a video, where you described the entire process in general. There are a lot of educational materials, not to say that they are bad, but they provide a lot of information in detail. Because of this overload, there is no general understanding of the process of automating analytics itself. And it seems that you do not understand anything, do not understand the sequence of actions. You made a cool video that clarifies this sequence. Thanks again.

@opeosibemekun5588 1 year ago

Very Helpful tutorial! thank you for sharing. Love how smooth the whole end to end process was.

@peterodedeyi3366 2 years ago

This work is worth more than rubies and gold. Keep it up, bro! Thank you.

@parry1454 1 year ago

Hi malvik, I was searching for a clear, informative tutorial for my team, and your video was exactly what we needed. It’s incredibly well-made, with rich content, clear step-by-step guidance, and excellent language. Thank you for your valuable contribution to the community and for helping so many of us!:yougotthis: