Sdf
SDF is a next-generation build system for data infrastructure. [Contact for Pricing]
Last verified:
What is Sdf?
SDF is a multi-dialect SQL compiler, transformation framework, and analytical database engine designed to be the fastest way to build a scalable, reliable, and optimized data warehouse. It natively compiles SQL dialects like Snowflake and connects to their corresponding data warehouses to materialize models. SDF aims to build the best end-to-end toolbox for data development, making developing data products like developing software products.
Key features include real-time impact analysis to prevent breaking changes from entering production, column-level lineage for full warehouse transparency, intelligent metadata with out-of-the-box guardrails to integrate business logic into code, and an in-process analytical database for on-premise data warehouses. SDF provides static analysis through sdf compile, lineage tracking via sdf lineage, and execution capabilities through sdf run. The tool uses a cache to fingerprint outputs and accelerate recomputation, located by default in the sdftarget/ directory.
SDF is designed for data professionals, data engineers, analytics engineers, developers, and students who work with SQL and data warehouses. It scales SQL understanding across organizations, allowing data teams to develop faster with timely error reporting and isolated environments. The tool integrates with dbt as a new transformation engine and accelerator, and supports all connectors and integrations for modern data stacks.
The platform validates SQL syntax and dependencies correctness instantly, running locally with lightning speed. SDF's execution capabilities are powered by DataFusion, and it supports Jinja templating with SDF SQL variables. Models can be materialized as tables, views, and more based on configuration, with each model receiving a fully qualified name (database.schema.table) that can correspond to a directory structure.
Sdf pricing
Pricing model: Freemium
SDF offers 4 pricing tiers with usage-based pricing on number of models. Developer plan is Always Free and includes SDF CLI Engine, Column-Level Lineage, Model Materialization, all connectors & integrations, Code Contracts & Tests, Local Execution (beta), and unlimited compile/test/runs for up to 200 models. Team plan costs $0.16/vCPU min and includes everything in Developer plus SDF Cloud Data Catalog, Git Impact Analysis, SDF Cloud Access, Automatic Warehouse Reports, Team Management & Access Control, and Slack Community Support for up to 450 models and 4 seats. Professional plan supports up to 1250 models with dedicated Slack support for growing data warehouses. Enterprise plan is custom-priced and includes everything in Team plus SSO integrations, enterprise-grade reliability & SLAs, prioritized feature development, and direct Slack connect support for warehouses over 1,000,000 models. Billing is monthly or annually with annual plans offered at discount. SDF Labs uses Stripe as billing partner.
Sdf pros
- Fastest way to build scalable and optimized data warehouse
- Multi-dialect SQL compiler supporting Snowflake and more
- Column-level lineage for full warehouse transparency
- Real-time impact analysis prevents breaking changes
- Unlimited compile, test, and runs on all plans
- Local execution with in-process analytical database
- Static analysis validates SQL syntax instantly
- Intelligent metadata with out-of-the-box guardrails
- Code contracts and tests for data quality
- Works offline with local compilation capability
- Powered by DataFusion for Rust performance
- Integrates seamlessly with dbt as transformation engine
- Smart caching accelerates recomputation
- Supports Jinja templating and SDF SQL variables
- Free Personal tier up to 200 models completely free
- All connectors and integrations included
- Git impact analysis for team collaboration
- Automatic warehouse reports generated
- Team management and access control available
- Works on Mac and Linux locally
Sdf cons
- No Windows support mentioned (Mac/Linux only)
- Local compilation requires advanced expertise
- Cloud features require paid Team plan
- Team plan priced at $0.16/vCPU min (usage-based)
- Enterprise plan requires custom quote (no transparent pricing)
- Annual billing discount details not specified
- Some features still in beta (Local Execution)
- SDF Cloud seats limited to 4 on Plus plan
- Professional plan capped at 1250 models
- Plus plan limited to 450 models
- No free tier for SDF Cloud (CLI only is free)
- Dedicated Slack support only on Professional+
- Documentation scattered across multiple pages
- Learning curve for workspace.sdf.yml configuration
- Country/region availability not clearly stated
Frequently asked questions about Sdf
What is SDF and what does it do?
SDF is a multi-dialect SQL compiler, transformation framework, and analytical database engine. It is the fastest way to build a scalable, reliable, and optimized data warehouse. SDF natively compiles SQL dialects like Snowflake and connects to their corresponding data warehouses to materialize models. It prevents breaking changes through real-time impact analysis, enables faster development with isolated environments, provides precise column-level lineage, and powers on-premise data warehouses with an in-process analytical database.
Is SDF free to use?
Yes, SDF has a free Developer plan that is Always Free. It includes the SDF CLI Engine, Column-Level Lineage, Model Materialization, all connectors & integrations, Code Contracts & Tests, Local Execution (beta), and unlimited compile, test, and runs. The free tier supports up to 200 models completely. SDF also provides free accounts to eligible non-commercial open source software initiatives.
How do I get started with SDF?
To get started, run the command 'sdf new --sample hello && cd hello' to create a new SDF Workspace. This creates a directory with a workspace.sdf.yml configuration file and a models/main.sql file. You can then run 'sdf compile --show all' to see static analysis results, and 'sdf run --show all' to execute queries. The Getting Started guide provides a 5-minute introduction, and there is a tutorial series using 'Mom's Flower Shop' workspace to learn real value.
What data warehouses does SDF support?
SDF natively compiles multiple SQL dialects including Snowflake. It supports all connectors and integrations for modern data stacks. The tool connects to corresponding data warehouses to materialize models. Specific integrations documented include Snowflake, BigQuery, AWS, DBT, CI/CD, Databricks, and Dagster.
How does SDF's column-level lineage work?
SDF guarantees rich column-level lineage through the 'sdf lineage' command. You can specify a particular column in a particular table to inspect its lineage. For example, 'sdf lineage main2 --column message' shows the copy relationship between columns. This provides full warehouse transparency and tracks how data flows through your models.
Can I compile SDF projects offline without a database?
Yes, SDF supports local compilation without running queries against your database. Use the '--prefer-local' flag with 'sdf compile' to compile entirely locally. This requires hydrating your workspace with schemas for remote sources locally in a 'sources' directory. Note that managing a locally compilable workspace requires significantly more maintenance and is recommended only for advanced users with strong SDF understanding.
What is the difference between SDF models and tables?
SDF refers to SQL statements as 'models'. Models differ from tables because they can be materialized as tables, views, and more based on configuration. Models can be templatized with Jinja and SDF SQL variables. SDF recommends specifying one model per file, as each model receives a fully qualified name (database.schema.table) that can correspond to a directory structure.
How does SDF caching work?
SDF uses a cache to fingerprint outputs and accelerate recomputation. The cache is located by default in the 'sdftarget/' directory and is machine-specific. It should not be checked into git (an appropriate .gitignore is created with 'sdf new'). SDF intelligently caches previous compilations, so already-compiled models are not re-compiled in subsequent runs, significantly speeding up compilation.
What support options are available for SDF?
Support options vary by plan. The Developer plan includes access to the Slack Community Support. The Professional plan includes dedicated Slack support. The Enterprise plan includes direct Slack connect support with prioritized feature development. There is also an SDF Community Slack at https://sdf.com/join where you can join thousands of data practitioners. Email support is available at [email protected].
How does SDF integrate with dbt?
dbt Labs acquired SDF Labs to bring SDF's SQL comprehension technology into dbt. SDF serves as a new transformation engine and dbt accelerator. With SDF integration under the hood, dbt becomes much faster and significantly more cost-efficient while unlocking new metadata use-cases like true column-level lineage. The two teams work together to integrate SDF capabilities into dbt for both dbt Core and dbt Cloud users.