Overview

SDF has the ability to connect with many databases, and supports numerous SQL dialects.

Are you looking for an integration that’s not yet supported? Let us know by giving us feebdack here

Use the below feature matrix to determine if SDF is right for you

StatusMeaning
🟒Supported
🟑Coming Soon
πŸ”΄Coming Later

Feature Matrix

Database Support

SDF is working to build parity with common data warehouse development patterns including Jinja macro pre-processing, materialization strategies, multiple environment support, and more.

FeatureSDF DBRedshiftSnowflakeBigQueryTrinoDatabricks
SQL Dialect Support🟒🟒🟒🟒🟒🟑
Column-level Lineage🟒🟒🟒🟒🟒🟒
Impact Analysis🟒🟒🟒🟒🟒🟑
Macros & Jinja🟒🟒🟒🟒🟒🟒
Zero-Copy Environments🟒🟒🟒🟒🟒🟑
Table ProviderπŸŸ‘πŸŸ’πŸŸ’πŸŸ’πŸ”΄πŸŸ‘
Run & ExecutionπŸŸ’πŸŸ‘πŸŸ’πŸŸ’πŸ”΄πŸŸ‘
Advanced MaterializationπŸŸ’πŸ”΄πŸŸ’πŸŸ‘πŸ”΄πŸŸ‘
Database AuthπŸŸ’πŸŸ’πŸŸ’πŸŸ’πŸ”΄πŸŸ‘
Local ExecutionπŸŸ’πŸ”΄πŸŸ‘πŸ”΄πŸŸ’πŸ”΄

When we say β€œSQL Dialect Support”, we mean support for SDF’s rich static analysis toolset - including compilation, lineage, dependency analysis, and more.

Authentication Support

SDF Supports authentication to one or more databases simultaneously through integrations & credentials.

To check the current status of all credentials held by SDF, run sdf auth status.

Authentication MethodSupported
AWS Keypair🟒
AWS SSO🟒
AWS ARN🟒
Snowflake User/Pass🟒
Snowflake SSO🟑
Snowflake Key Pair🟒
BigQuery Service Accounts🟒
BigQuery Application Default CredentialsπŸ”΄
Redshift JDBCπŸ”΄
Trino JDBCπŸ”΄
Trino JDBCπŸ”΄
Spark JDBCπŸ”΄

Orchestration Support

SDF Supports simple orchestration. Commands like compile & run are executed in dependency order, allowing for simple orchestration via cron jobs, or GitHub Actions.

For more advanced orchestration needs, SDF recommends Dagster.

OrchestratorSupportedDocumentation
Github🟒here
Dagster🟒here
Airflow🟑Coming Soon

Platforms

SDF is currently supported on the following platforms. The executable is small (<75mb), but heavily parallelized. As such, SDF’s resource consumption is dependent entirely on your data processing requirements.

PlatformsSupported
Linux Intel X86-64βœ”οΈ
Linux Arm AARCH-64βœ”οΈ
Apple Intel X86-64βœ”οΈ
Apple Arm AARCh-64βœ”οΈ
Windows Intel X86-64❌
Web-Assembly❌

For a complete list of releases, please see the official GitHub releases page.

SDF is hardware-optimized. For best performance, download the version that matches your chipset.