Features Matrix
SDF is a work in progress. The data ecosystem contains many tools, databases, and more to integrate with.
Overview
SDF has the ability to connect with many databases, and supports numerous SQL dialects.
Are you looking for an integration that’s not yet supported? Let us know by giving us feebdack here
Use the below feature matrix to determine if SDF is right for you
| Status | Meaning |
|---|---|
| 🟢 | Supported |
| 🟡 | Coming Soon |
| 🔴 | Coming Later |
Feature Matrix
Database Support
SDF is working to build parity with common data warehouse development patterns including Jinja macro pre-processing, materialization strategies, multiple environment support, and more.
| Feature | SDF DB | Redshift | Snowflake | BigQuery | Trino | Databricks |
|---|---|---|---|---|---|---|
| SQL Dialect Support | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟡 |
| Column-level Lineage | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| Impact Analysis | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟡 |
| Macros & Jinja | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| Zero-Copy Environments | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟡 |
| Table Provider | 🟡 | 🟢 | 🟢 | 🟢 | 🔴 | 🟡 |
| Run & Execution | 🟢 | 🟡 | 🟢 | 🟢 | 🔴 | 🟡 |
| Advanced Materialization | 🟢 | 🔴 | 🟢 | 🟡 | 🔴 | 🟡 |
| Database Auth | 🟢 | 🟢 | 🟢 | 🟢 | 🔴 | 🟡 |
| Local Execution | 🟢 | 🔴 | 🟡 | 🔴 | 🟢 | 🔴 |
When we say “SQL Dialect Support”, we mean support for SDF’s rich static analysis toolset - including compilation, lineage, dependency analysis, and more.
Authentication Support
SDF Supports authentication to one or more databases simultaneously through integrations & credentials.
To check the current status of all credentials held by SDF, run sdf auth status.
| Authentication Method | Supported |
|---|---|
| AWS Keypair | 🟢 |
| AWS SSO | 🟢 |
| AWS ARN | 🟢 |
| Snowflake User/Pass | 🟢 |
| Snowflake SSO | 🟡 |
| BigQuery Service Accounts | 🟡 |
| BigQuery Application Default Credentials | 🔴 |
| Redshift JDBC | 🔴 |
| Trino JDBC | 🔴 |
| Trino JDBC | 🔴 |
| Spark JDBC | 🔴 |
Orchestration Support
SDF Supports simple orchestration. Commands like compile & run are executed in dependency order, allowing for simple orchestration via cron jobs, or GitHub Actions.
For more advanced orchestration needs, SDF recommends Dagster.
| Orchestrator | Supported | Documentation |
|---|---|---|
| Github | 🟢 | here |
| Dagster | 🟢 | here |
| Airflow | 🟡 | Coming Soon |
Platforms
SDF is currently supported on the following platforms. The executable is small (<75mb), but heavily parallelized. As such, SDF’s resource consumption is dependent entirely on your data processing requirements.
| Platforms | Supported |
|---|---|
| Linux Intel X86-64 | ✔️ |
| Linux Arm AARCH-64 | ✔️ |
| Apple Intel X86-64 | ✔️ |
| Apple Arm AARCh-64 | ✔️ |
| Windows Intel X86-64 | ❌ |
| Web-Assembly | ❌ |
For a complete list of releases, please see the official GitHub releases page.
SDF is hardware-optimized. For best performance, download the version that matches your chipset.