Data Engineering & Platforms
Lakehouse Platforms and Delta Lake
D02 · Practitioner
Entry requirements
D01; Python and SQL proficiency
What 36 hours means
- Lessons
- 24 h
- Capstone
- 8 h
- Assessment
- 2 h
- Remediation
- 2 h
Syllabus
Four modules, twelve lessons. Every lesson states what it covers and the evidence you produce for it.
1Platform choices3 lessons · 6 h
- D02.01
Data warehouse and lakehouse
120 minCompare storage and compute patterns for a telemetry workload.
Evidence: An architecture decision records cost, latency and governance needs.
- D02.02
Spark dataframes
120 minLoad and transform partitioned data with explicit schemas.
Evidence: Output schema and aggregates match provided checks.
- D02.03
Storage layout
120 minChoose file format and partition boundaries for common queries.
Evidence: A plan avoids excessive tiny files and high-cardinality partitions.
2Delta tables3 lessons · 6 h
- D02.04
Delta transactions
120 minCreate a table and inspect transaction history.
Evidence: Two controlled writes produce the expected table state.
- D02.05
Updates and merge
120 minApply corrected events using stable business keys.
Evidence: Repeated merges are idempotent and preserve unaffected records.
- D02.06
Schema evolution
120 minTest additive changes and incompatible type changes.
Evidence: A migration passes compatibility tests or rejects unsafe input.
3Governance3 lessons · 6 h
- D02.07
Historical versions
120 minQuery earlier table state and plan retention implications.
Evidence: A historical result is reproduced with its version recorded.
- D02.08
Quality layers
120 minImplement raw, validated and business-ready transformations.
Evidence: Rejected records remain inspectable with reason codes.
- D02.09
Governance concepts
120 minDesign catalog, schema, table and role permissions.
Evidence: A permissions matrix enforces least privilege for three roles.
4Production design3 lessons · 6 h
- D02.10
Unity Catalog guided practice
120 minMap the governance design to a licensed workspace if supplied; otherwise use a clearly labelled simulation.
Evidence: A permission test log identifies which actions were simulated.
- D02.11
Optimisation and observation
120 minMeasure table layout and query changes on the same workload.
Evidence: Before-and-after measurements include correctness checks.
- D02.12
Migration and recovery
120 minPlan a cutover, rollback and historical reconciliation.
Evidence: A migration checklist includes data checks and a rollback trigger.
Capstone
Build a bronze, silver and gold lakehouse for synthetic telemetry, including schema evolution, historical correction and governed access.
Lab environment
Data sandbox; optional licensed vendor workspace
Lab infrastructure is not yet provisioned, so this describes the environment the course is designed for rather than one you can launch today.