dbt and Data Pipelines

ULC-014 builds the Riverbank Media pipeline with hand-written SQL and a small Python runner. That is the right way to learn what a pipeline does, and the wrong way to maintain fifty models with a team. This elective rebuilds the *transformation* half of that same pipeline with **dbt-core 1.11** and the **dbt-postgres** adapter, on the same `riverbank` database, so you can see exactly what dbt adds — and prove it, because the pipeline's own mart is still sitting there to compare against. You install dbt into a virtual environment from a pinned offline wheelhouse, declare the pipeline's clean layer as sources, and build staging views and mart tables with `source()` and `ref()`. You make the fact table incremental and prove a second run inserts nothing. You snapshot a dimension, apply a change set to the source export, re-ingest, and watch the snapshot close a version and open a new one. You test every key and relationship, plant a fault that turns a test red, read the stored failures, fix it and watch it go green. You document every model and column, generate the lineage graph, and declare the exposure that says who reads the mart. Then you put it in a pipeline. You extend ULC-014's runner to call `dbt build` and record the phase in its load log — through a written change record with a rollback you have executed. You add a production target in its own schema, write a CI script, and prove it fails closed by planting a model that cannot compile. You measure a mart query, add post-hook indexes, and make the incremental strategy explicit. You enforce a contract on the fact table and configure source freshness. Every command, model, test and number in these lessons was executed on PostgreSQL 16 with dbt-core 1.11.15 and dbt-postgres 1.11.0, and the output you see is the output that was observed. The final project is an engagement: Riverbank Media asks you, as its first analytics engineer, to deliver and hand over the transformation layer — graded by automated checks against the warehouse your models actually built, and by a rubric against your handover.