SDF is a work in progress. The data ecosystem contains many tools, databases, and more to integrate with.
Status | Meaning |
---|---|
π’ | Supported |
π‘ | Coming Soon |
π΄ | Coming Later |
Feature | SDF DB | Redshift | Snowflake | BigQuery | Trino | Databricks |
---|---|---|---|---|---|---|
SQL Dialect Support | π’ | π’ | π’ | π’ | π’ | π‘ |
Column-level Lineage | π’ | π’ | π’ | π’ | π’ | π’ |
Impact Analysis | π’ | π’ | π’ | π’ | π’ | π‘ |
Macros & Jinja | π’ | π’ | π’ | π’ | π’ | π’ |
Zero-Copy Environments | π’ | π’ | π’ | π’ | π’ | π‘ |
Table Provider | π‘ | π’ | π’ | π’ | π΄ | π‘ |
Run & Execution | π’ | π‘ | π’ | π’ | π΄ | π‘ |
Advanced Materialization | π’ | π΄ | π’ | π‘ | π΄ | π‘ |
Database Auth | π’ | π’ | π’ | π’ | π΄ | π‘ |
Local Execution | π’ | π΄ | π‘ | π΄ | π’ | π΄ |
sdf auth status
.
Authentication Method | Supported |
---|---|
AWS Keypair | π’ |
AWS SSO | π’ |
AWS ARN | π’ |
Snowflake User/Pass | π’ |
Snowflake SSO | π‘ |
Snowflake Key Pair | π’ |
BigQuery Service Accounts | π’ |
BigQuery Application Default Credentials | π΄ |
Redshift JDBC | π΄ |
Trino JDBC | π΄ |
Trino JDBC | π΄ |
Spark JDBC | π΄ |
compile
& run
are executed in dependency order, allowing for simple orchestration via cron jobs, or GitHub Actions.
For more advanced orchestration needs, SDF recommends Dagster.
Orchestrator | Supported | Documentation |
---|---|---|
Github | π’ | here |
Dagster | π’ | here |
Airflow | π‘ | Coming Soon |
Platforms | Supported |
---|---|
Linux Intel X86-64 | βοΈ |
Linux Arm AARCH-64 | βοΈ |
Apple Intel X86-64 | βοΈ |
Apple Arm AARCh-64 | βοΈ |
Windows Intel X86-64 | β |
Web-Assembly | β |