technifyed

Microsoft via Microsoft Learn

Design and implement data pipelines with Azure Databricks

Overview

Learn how to design and implement robust data pipelines using Lakehouse architecture principles, medallion architecture, and Lakeflow pipelines in Azure Databricks.

Syllabus 10

  1. Introduction 5 minutes
  2. Design order of operations for a pipeline 5 minutes
  3. Choose notebook vs Lakeflow Pipelines 5 minutes
  4. Design Lakeflow job logic 10 minutes
  5. Design error handling in pipelines and jobs 10 minutes
  6. Create pipeline with notebook 5 minutes
  7. Create pipeline with Lakeflow pipelines 5 minutes
  8. Exercise - Design and Implement Data Pipelines with Azure Databricks 45 minutes
  9. Module assessment 5 minutes
  10. Summary 5 minutes

Skills you'll practise

Azure Databricks

Advantages and disadvantages

Advantages

  • Free, official and kept up to date by Microsoft.
  • Free sandbox environments for many Azure exercises, no credit card needed.
  • Counts toward Microsoft certification exams (AZ-900, AI-900, PL-300…).
  • From Microsoft, a well-regarded name.
  • Free digital badge you can add to LinkedIn.
  • Completely free.
  • Self-paced: start any time.
  • A clear syllabus (10 parts) you can see before you start.

Disadvantages

  • Mostly reading, not video.
  • Focused on Microsoft products.

Some points apply to every course of this kind; see how we rank.

Free, with a free credential

  • Free: Sign in with any Microsoft account.
  • Badge: Finish it to earn a free badge on your Microsoft Learn profile.

Microsoft is in Tier 2: excellent universities and the companies that build the technology of our institution ranking (86/100).

Similar courses

Compare these

Massachusetts Institute of Technology · YouTube

MIT RES.LL-005 Mathematics of Big Data and Machine Learning, IAP 2020

Instructor: Jeremy Kepner, Vijay Gadepally View the complete course: https://ocw.mit.edu/RES-LL-005IAP20 YouTube Playlist: https://www.youtube.com/playlist?list... This January 2020 IAP class focused on artificial intelligence and machine learning with an emphasis on the data handling challenges. L…

Free video20 videos, 14 hours

Stanford University · YouTube

Stanford Webinar: When Your Big Data Seems Too Small

A Stanford Webinar presented by: Stanford's Databases and the Foundations in Computer Science graduate certificate programs "When Your Big Data Seems Too Small - Accurate Inferences Beyond the Empirical Distribution" Speaker: Gregory Valiant, Stanford University Many of the techniques and algorithm…

Free video50 minutes

Stanford University · YouTube

CS545 - Information and Data Analytics Seminar Series

This seminar features leading Industrial and academic experts on big data analytics, information management, data mining, machine learning, and large-scale data processing.

Free video5 videos, 4.5 hours

Massachusetts Institute of Technology · YouTube

MIT D4M: Signal Processing on Databases, Fall 2012

View the complete course: http://ocw.mit.edu/RESLL-005F12 Instructor: Jeremy Kepner This course focused on the singal processing on databases, based on detection theory and linear algebra with databases. By bringing together the concepts of strings and searching, student applied the big data analys…

Free video18 videos, 12 hours

Crash Course · YouTube

Statistics

In 44 episodes, Adriene Hill teaches you Statistics! This course is based on the 2018 AP Statistics curriculum and introduces everything from basic descriptive statistics to data collection to hot topics in data analysis like Big Data and neural networks. By the end of the course, you will be able…

Free video45 videos, 9 hours

freeCodeCamp · YouTube

Azure Administrator Certification (AZ-104) - Full Course to PASS the Exam

The Azure Administrator is the most popular Azure certification because it provides the broadest amount of practical knowledge to manage Azure infrastructure and it allows you to pivot into any career path for Azure-based roles. By the end of this full course taught by an expert trainer, you will b…

Free video11 hours

Enter your email and the official page opens. Phone is optional.