HTTP APIs for Data Engineering

Overview

What you'll learn

• Build typed DataFrames with the Polars Rust API and load CSV data with explicit types
• Choose between eager and lazy evaluation, and read a LazyFrame's plan before calling collect
• Clean a real dataset with documented drop, fill and flag rules, and filter out-of-range values
• Sort, group, aggregate and join DataFrames, and use a left join's nulls as a data-quality signal
• Assemble a bronze, silver and gold medallion pipeline as a Rust command-line tool
• Serve the gold layer over HTTP with Axum, mapping query parameters onto Polars expressions safely

Syllabus 4

1. Polars Foundations : why Polars, DataFrames and Series, reading CSV, casting, and eager versus lazy evaluation
2. Cleaning and Transforming Wine Data : nulls, normalization, filtering, sorting, grouping, and join types for enrichment
3. Building the Medallion Pipeline : the guarantees of bronze, silver and gold, built as a single command-line tool
4. Serving the Gold Layer : an Axum API over the gold LazyFrame, with filter, group-by, search and region routes, and a graded serve project

Advantages and disadvantages

Some points apply to every course of this kind; see how we rank.

Before you start

Comfort reading and writing basic Rust: functions, structs, Result and ?. No prior Polars, pandas or web-framework experience is assumed. To run the examples you need a terminal and a Rust toolchain. The dataset is a public CSV that the labs download in one command.