Building the Modern Data Stack from Scratch

The modern data stack has matured into a set of well-understood patterns, but assembling them into a cohesive platform still requires deliberate decisions. In this post I walk through how I approach building a stack from the ground up.

At the ingestion layer, the goal is to centralize raw data with minimal transformation. Managed connectors handle the bulk of the work, while custom extractors fill the gaps for niche sources. Keeping raw data immutable gives you a reliable foundation to rebuild downstream models whenever logic changes.

Transformation is where dbt earns its place. By treating SQL as software, complete with version control, testing, and documentation, you create a maintainable layer that the entire team can reason about. Modular models, clear naming conventions, and incremental builds keep things fast as data volumes grow.

Finally, the serving layer connects models to the people who need them. Whether that's a BI tool, a reverse-ETL sync, or an API, the principle stays the same: expose clean, well-tested, well-documented data products rather than ad hoc queries.