dbt: Data transformation using software engineering practices
SQL-based transformation framework for analytics data warehouses.
Learn more about dbt-core
dbt is a Python-based tool that compiles SQL select statements into executable transformations for data warehouses. It operates on a directed acyclic graph (DAG) model where each SQL query represents a node, and dbt automatically determines execution order based on declared dependencies between models. The tool supports multiple data warehouse backends and provides features for testing data quality, generating documentation, and version controlling transformation logic. Common use cases include building data marts, creating dimensional models, and establishing repeatable ELT (Extract, Load, Transform) pipelines in analytics workflows.
SQL-first approach
dbt uses standard SQL select statements as the primary interface for defining transformations, rather than requiring a proprietary language or visual interface. This allows analysts familiar with SQL to define complex data pipelines without learning additional frameworks.
Dependency management via DAG
dbt automatically constructs a directed acyclic graph from declared model relationships and determines the correct execution order. This eliminates manual scheduling logic and allows developers to reference upstream models using the ref() function rather than hardcoding table names.
Built-in testing and documentation
dbt includes native support for data quality tests and automatic documentation generation from model definitions and comments. These capabilities are integrated into the project structure and CI/CD workflows rather than requiring separate tools.
-- models/orders_summary.sql
-- Reference a staging model to build aggregated orders
select
customer_id,
count(*) as order_count,
sum(order_total) as total_spent
from {{ ref('stg_orders') }}
group by customer_idMaintenance release fixing packaging; no user-facing changes or breaking updates.
- –Packaging fix ensures dbt/jsonschemas directory is included in distributions; no action required.
- –Release notes do not specify breaking changes, new requirements, or security fixes.
Patch release fixing partial parsing errors with versioned models; no breaking changes or new requirements.
- –Upgrade to resolve partial parsing crashes when using versioned models in your project.
- –Internal JSON schema directory reorganization has no user-facing impact.
Beta adds function volatility specs and Python UDF parsing; function node `type` moved to config (may break existing function definitions).
- –Update function node definitions: `type` now lives in config block, and function configs can be set in dbt_project.yml.
- –Upgrade dbt-semantic-interfaces to 0.10.x if using semantic layer features; release notes do not specify compatibility impact.
Top in Data Engineering
Related Repositories
Discover similar tools and frameworks used by developers
pdfplumber
Python library for extracting PDF text and tables.
n8n
Node-based automation platform with JavaScript and Python scripting.
docling
Fast document parser for RAG and AI workflows.
exceljs
Parse, modify, and generate XLSX files in Node.js.
flyway
Version-controlled SQL migrations with automated execution tracking.