Installation
SDF is a data development framework for building scalable data warehouses written in Rust. No JVM required
Installing SDF
SDF is installed as a single, dependency free binary on a host machine. The install script will first check your operating system and architecture, then install the appropriate latest stable version of SDF and its dependencies for your system.
The sdf command will also be added to your path.
To the latest stable release of SDF:
Verify Installation
To verify that SDF is installed correctly, run sdf --help
SDF: A fast SQL compiler, local development framework, and in-memory analytical database
Usage: sdf [OPTIONS] <COMMAND>
Commands:
new Create a new sdf workspace
clean Remove artifacts that sdf has generated in the past
compile Compile models
lint Lint models
format Format models
lsp1 Lsp wave1
run Run models
build Run model and tests, and if successful, publish, aka “write-audit-publish”
test Test your models
stats Statistics for your data
report Report code quality
check Check code quality
lineage Display lineage for a given table and/or column
push Push a local workspace to the SDF Service
system System maintenance, install and update
auth Authenticate CLI to services like SDF, AWS, OpenAI, etc
man Display reference material, like the CLI, dialect specific functions, schemas for authoring and interchange
init Initialize a workspace interactively
dbt Initialize an sdf workspace from an existing dbt project
exec Execute custom scripts
help Print this message or the help of the given subcommand(s)
Options:
—log-level <LOG_LEVEL> Set log level [possible values: trace, debug, debug-pretty, info, warn, error]
—log-file <LOG_FILE> Creates or replaces the log file
—show-all-errors Don’t suppress errors
-h, —help Print help
-V, —version Print version
Use an editor like Visual Studio Code for better auto-complete and development hints.
YML Extension For VSCode
If you’re using VSCode, please download the official Red Hat YML extension which can provide intelisesnse support for SDF’s official YML schema. This extension will show configuration errors, options, and more.
The JSON schemas for SDF’s .sdf.yml extension are published on the schema store and in the sdf-cli repository.
SDF Versions and Updating
From time to time, it may be necessary to update the CLI to get the latest and greatest features, or to connect to SDF Cloud Services.
If you’ve already installed SDF, you can update to the latest version by running the command below. By default, this will update to the latest version within your current release channel (stable or preview).
You can also update to a specific release channel: To view the available versions please see the github releases page.
You can check the currently installed version of your CLI with:
Release Tracks
SDF has two release tracks: stable and preview
stable (X.X.X-s) -> receives only explicit bug fixes, and is updated monthly
preview (X.X.X-p) -> receives new features multiple times per week
Installing a Release Channel Directly
You may install a preview version of SDF directly:
Switching Release Channels
Any SDF installation may switch tracks by updating into a different track, or version.
To switch channels while updating, use the --stable or --preview flags:
Preview versions offer early access to new features but may be less stable. Use for testing and feedback.
Uninstalling SDF
If you ever need to uninstall SDF, simply run
Info
SDF is currently supported on the following platforms. The executable is small, but heavily parallelized. As such, SDF’s resource consumption is dependent entirely on your data processing requirements.
| Platforms | Supported |
|---|---|
| Linux Intel X86-64 | ✔️ |
| Linux Arm AARCH-64 | ✔️ |
| Apple Intel X86-64 | ✔️ |
| Apple Arm AARCh-64 | ✔️ |
| Windows Intel X86-64 | ❌ |
| Web-Assembly | ❌ |
For a complete list of releases, please see the official github releases page.
SDF is hardware optimized. For best performance, download the version that matches your chipset.