Features Matrix
SDF is a work in progress. The data ecosystem contains many tools, databases, and more to integrate with.
Overview
SDF has the ability to connect with many databases, and supports numerous SQL dialects.
Are you looking for an integration thatβs not yet supported? Let us know by giving us feebdack here
Use the below feature matrix to determine if SDF is right for you
Status | Meaning |
---|---|
π’ | Supported |
π‘ | Coming Soon |
π΄ | Coming Later |
Feature Matrix
Database Support
SDF is working to build parity with common data warehouse development patterns including Jinja macro pre-processing, materialization strategies, multiple environment support, and more.
Feature | SDF DB | Redshift | Snowflake | BigQuery | Trino | Databricks |
---|---|---|---|---|---|---|
SQL Dialect Support | π’ | π’ | π’ | π’ | π’ | π‘ |
Column-level Lineage | π’ | π’ | π’ | π’ | π’ | π’ |
Impact Analysis | π’ | π’ | π’ | π’ | π’ | π‘ |
Macros & Jinja | π’ | π’ | π’ | π’ | π’ | π’ |
Zero-Copy Environments | π’ | π’ | π’ | π’ | π’ | π‘ |
Table Provider | π‘ | π’ | π’ | π’ | π΄ | π‘ |
Run & Execution | π’ | π‘ | π’ | π’ | π΄ | π‘ |
Advanced Materialization | π’ | π΄ | π’ | π‘ | π΄ | π‘ |
Database Auth | π’ | π’ | π’ | π’ | π΄ | π‘ |
Local Execution | π’ | π΄ | π‘ | π΄ | π’ | π΄ |
When we say βSQL Dialect Supportβ, we mean support for SDFβs rich static analysis toolset - including compilation, lineage, dependency analysis, and more.
Authentication Support
SDF Supports authentication to one or more databases simultaneously through integrations & credentials.
To check the current status of all credentials held by SDF, run sdf auth status
.
Authentication Method | Supported |
---|---|
AWS Keypair | π’ |
AWS SSO | π’ |
AWS ARN | π’ |
Snowflake User/Pass | π’ |
Snowflake SSO | π‘ |
BigQuery Service Accounts | π‘ |
BigQuery Application Default Credentials | π΄ |
Redshift JDBC | π΄ |
Trino JDBC | π΄ |
Trino JDBC | π΄ |
Spark JDBC | π΄ |
Orchestration Support
SDF Supports simple orchestration. Commands like compile
& run
are executed in dependency order, allowing for simple orchestration via cron jobs, or GitHub Actions.
For more advanced orchestration needs, SDF recommends Dagster.
Orchestrator | Supported | Documentation |
---|---|---|
Github | π’ | here |
Dagster | π’ | here |
Airflow | π‘ | Coming Soon |
Platforms
SDF is currently supported on the following platforms. The executable is small (<75mb), but heavily parallelized. As such, SDFβs resource consumption is dependent entirely on your data processing requirements.
Platforms | Supported |
---|---|
Linux Intel X86-64 | βοΈ |
Linux Arm AARCH-64 | βοΈ |
Apple Intel X86-64 | βοΈ |
Apple Arm AARCh-64 | βοΈ |
Windows Intel X86-64 | β |
Web-Assembly | β |
For a complete list of releases, please see the official GitHub releases page.
SDF is hardware-optimized. For best performance, download the version that matches your chipset.