Command-Line Help for sdf

This document contains the help content for the sdf command-line program.

Command Overview:

sdf

SDF: A fast SQL compiler, local development framework, and in-memory analytical database

Usage: sdf [OPTIONS] COMMAND

Subcommands:
  • new — Create a new sdf workspace
  • clean — Remove artifacts that sdf has generated in the past
  • compile — Compile models
  • run — Run models
  • test — Test your models
  • stats — Statistics for your data
  • report — Report code quality
  • check — Check code quality
  • lineage — Display lineage for a given table and/or column
  • push — Push a local workspace to the SDF Service
  • system — System maintenance, install and update
  • auth — Authenticate CLI to services like SDF, AWS, OpenAI, etc
  • man — Display reference material, like the CLI, dialect specific functions, schemas for authoring and interchange
  • dbt — Initialize an sdf workspace from an existing dbt project
  • exec — Execute custom scripts
Options:
  • --no-parallel — Do NOT run in parallel

    Possible values: true, false

  • --num-threads NUM_THREADS — Number of threads to use for parallel operations. If unset, default is number of CPU cores. Must be greater than 0 if set

  • --stack-size STACK_SIZE — Set stack size for worker thread, in bytes

  • --log-level LOG_LEVEL — Set log level

    Possible values: trace, debug, info, warn, error

  • --log-file LOG_FILE — Creates or replaces the log file

  • --log-form LOG_FORM — Set the log format; default flat

    Possible values:

    • flat: Writes nd-json, compact, no nested objects
    • nested: Writes nd-json, compact, nested objects
    • pretty: Writes nd-json, pretty, nested objects
  • --log-filter LOG_FILTER — Filters log messages; filters are written as key : prefix_of_value

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf new

Create a new sdf workspace

Usage: sdf new [OPTIONS] [PATH]

Arguments:
  • PATH — Create a new sdf workspace at path
Options:
  • --list-samples — List all available samples

    Default value: false

    Possible values: true, false

  • --sample SAMPLE — Create a workspace with the sample content at path

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

  • -s, --show SHOW — Display progress messages

    Default value: progress

    Possible values: progress, none

sdf clean

Remove artifacts that sdf has generated in the past

Usage: sdf clean [OPTIONS]

Options:
  • -e, --environment ENVIRONMENT

  • --path PATH — Remove artifacts at path that sdf has created in the past [default: current workspace directory]

  • -s, --show SHOW — Display progress messages

    Default value: progress

    Possible values: progress, none

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf compile

Compile models

Usage: sdf compile [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Compile only the given source dirs, files or tables [default: all models]
Options:
  • --infer-functions — Infer function signature

    Default value: false

    Possible values: true, false

  • --infer-tables — Infer table schema and generate ddls

    Default value: false

    Possible values: true, false

  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

  • --no-incremental-mode — Set incremental-mode to false (deault is true)

    Default value: false

    Possible values: true, false

  • --no-snapshot-mode — Set snapshot-mode to false (deault is true)

    Default value: false

    Possible values: true, false

sdf run

Run models

Usage: sdf run [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Run only the given source dirs, files or tables [default: all models]
Options:
  • --exec-raw-query — Hidden Option to Run Queries in Raw Mode

    Default value: false

    Possible values: true, false

  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • -d, --date DATE — Run command for this date , use YYYY-MM-DD or YYYY-MM-DDTHH:MM format

  • --from FROM — Run command for all dates from from (inclusive), use YYYY-MM-DD or YYYY-MM-DDTHH:MM format

  • --to TO — Run command for all dates to to (exclusive), use YYYY-MM-DD or YYYY-MM-DDTHH:MM format [default: now]

  • --dry-run — Plan command but don’t evaluate it

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf test

Test your models

Usage: sdf test [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Assess code (data) quality: use source dirs, files or tables to determine which code (data) contracts to run [default: all code (data) contracts]
Options:
  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • -u, --sql-logic-test-update — Goldie update the sqllogictests files

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf stats

Statistics for your data

Usage: sdf stats [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Profile data quality: use source dirs, files or tables to determine which data statistics to run [default: all data stats]
Options:
  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf report

Report code quality

Usage: sdf report [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Profile code (data) quality, use source dirs, files or tables to determine which code (data) reports to run [default: all code (data) reports]
Options:
  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf check

Check code quality

Usage: sdf check [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Check code (data) quality: use source dirs, files or tables to determine which code (data) checks to run [default: all code (data) checks]
Options:
  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf lineage

Display lineage for a given table and/or column

Usage: sdf lineage [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Target selection: compute the lineage for the given source dirs, files or @tables [default: all models]
Options:
  • --column COLUMN — The column for which to compute lineage

  • --forward — Compute downstream lineage instead of the default upstream lineage

    Default value: false

    Possible values: true, false

  • --show-scans — Display scan dependencies in addition to copy and mod dependencies. Unset by default

    Default value: false

    Possible values: true, false

  • --max-depth MAX_DEPTH — Limiting the depth of shown lineage tree. Default value of 0 shows full lineage

    Default value: 0

  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf push

Push a local workspace to the SDF Service

Usage: sdf push [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Target selection: push only the given source dirs, files or tables [default: current workspace directory]
Options:
  • --delete — Delete this workspace from the SDF Service

    Default value: false

    Possible values: true, false

  • -y, --yes — Answer yes to all prompts which include credentials’ uploading to the console when using a table provider

    Default value: false

    Possible values: true, false

  • -d, --dry-run — No changes will be made to the console when this is set

    Default value: false

    Possible values: true, false

  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf system

System maintenance, install and update

Usage: sdf system [OPTIONS] COMMAND

Subcommands:
  • update — Update sdf in place to the latest version
  • uninstall — Uninstall sdf from the system
Options:
  • -s, --show SHOW — Display messages

    Default value: progress

    Possible values: progress, none

sdf system update

Update sdf in place to the latest version

Usage: sdf system update

Options:
  • --version VERSION — Update sdf to this version [default: latest version]

sdf system uninstall

Uninstall sdf from the system

Usage: sdf system uninstall

sdf auth

Authenticate CLI to services like SDF, AWS, OpenAI, etc

Usage: sdf auth [OPTIONS] COMMAND

Subcommands:
  • login
  • logout — Log out of SDF Service
  • status — Show status of credentials / tokens
Options:
  • -e, --environment ENVIRONMENT — Use this environment

  • -s, --show SHOW — Display messages [default: progress if TARGETS is empty, all otherwise]

    Possible values: all, progress, result, none

  • -q, --query QUERY — Supply a .sql file or provide a sql snippet on the cmd line, e.g. ‘select * from t’

  • --stage STAGE — Run the following stages [default: all stages]

    Possible values: preprocess, parse, resolve, classify, execute

  • --cache CACHE — Controls cache use

    Default value: read-write

    Possible values: read-write, write-only, read-only, none

  • --no-cache — Shorthand for cache policy none

    Default value: false

    Possible values: true, false

  • --fetch — Fetches changes to tables with remote origin

    Default value: false

    Possible values: true, false

  • --save-assembly — Saves assembly.sdf.yml and sdf.information_schema.* in sdftarget/profile/…

    Default value: false

    Possible values: true, false

  • --targets-only — Processes only specified targets assuming that all the non-target dependencies already exist

    Default value: false

    Possible values: true, false

  • --show-assembly — ONLY FOR TEST: Shows assembly.sdf.yml on console

    Default value: false

    Possible values: true, false

  • --show-preprocessed — ONLY FOR TEST: Shows preprocessed .sql on console

    Default value: false

    Possible values: true, false

  • --save-preprocessed — ONLY FOR TEST: Save preprocessed .sql in compiled dir

    Default value: false

    Possible values: true, false

  • --vars VARS — Supply var bindings as a yml file or provide them as string e.g. ‘(key: value)’

  • -x, --experimental — Modern output

    Default value: false

    Possible values: true, false

  • --downstream — Execute cmd not only the given targets but also for all its downstream artifacts

    Default value: false

    Possible values: true, false

  • --format FORMAT — Show error tables in this format

    Default value: table

    Possible values: table, csv, tsv, json, nd-json, yml

  • --limit LIMIT — Limiting number of shown rows. Run with —limit 0 to remove limit

  • --no-propagate — Do NOT perform propagate phase, short form of —stage resolve

    Default value: false

    Possible values: true, false

  • --no-scripts — Do not run scripts

    Default value: false

    Possible values: true, false

  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

sdf auth login

Usage: sdf auth login [OPTIONS] [COMMAND]

Subcommands:
  • aws — Configure how to authenticate with AWS
  • snowflake — Configure how to authenticate with Snowflake
Options:
  • -n, --name NAME — Name of the credential to use
  • --id-token ID_TOKEN — Path to a file containing an OIDC identity token (a JWT)
  • --access-key ACCESS_KEY — Access key for headless authentication
  • --secret-key SECRET_KEY — Secret key for headless authentication
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials will be stored (default is platform specific)

sdf auth login aws

Configure how to authenticate with AWS

Usage: sdf auth login aws [OPTIONS]

Options:
  • -n, --name NAME — Name of the credential to use

  • --default-region DEFAULT_REGION — AWS Region to use (default: us-east-1)

    Default value: us-east-1

  • --profile PROFILE — AWS profile to use, as usually defined in ~/.aws/config or ~/.aws/credentials

  • --role-arn ROLE_ARN — ARN of the role to assume

  • --external-id EXTERNAL_ID — External ID to use when assuming the role

  • --use-web-identity — Use web identity to authenticate

    Default value: false

    Possible values: true, false

  • --bucket-region-map BUCKET_REGION_MAP — Mapping of bucket names to regions

  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials will be stored (default is platform specific)

  • --access-key-id ACCESS_KEY_ID — AWS access key id

  • --secret-access-key SECRET_ACCESS_KEY — AWS secret access key

  • --session-token SESSION_TOKEN — AWS session token

sdf auth login snowflake

Configure how to authenticate with Snowflake

Usage: sdf auth login snowflake [OPTIONS] --account-id ACCOUNT_ID --username USERNAME

Options:
  • -n, --name NAME — Name of the credential to use
  • -a, --account-id ACCOUNT_ID — Snowflake account id
  • -U, --username USERNAME — Snowflake username
  • -P, --password PASSWORD — Snowflake password
  • -r, --role ROLE — Snowflake role
  • -W, --warehouse WAREHOUSE — Snowflake warehouse
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials will be stored (default is platform specific)

sdf auth logout

Log out of SDF Service

Usage: sdf auth logout [OPTIONS] [COMMAND]

Subcommands:
  • aws — Logout from AWS
  • openai — Logout from OpenAI
  • snowflake — Logout from Snowflake
Options:
  • -n, --name NAME — Name of the credential to use
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials are be stored (default is platform specific)

sdf auth logout aws

Logout from AWS

Usage: sdf auth logout aws [OPTIONS]

Options:
  • -n, --name NAME — Name of the credential to use
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials are be stored (default is platform specific)

sdf auth logout openai

Logout from OpenAI

Usage: sdf auth logout openai [OPTIONS]

Options:
  • -n, --name NAME — Name of the credential to use
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials are be stored (default is platform specific)

sdf auth logout snowflake

Logout from Snowflake

Usage: sdf auth logout snowflake [OPTIONS]

Options:
  • -n, --name NAME — Name of the credential to use
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials are be stored (default is platform specific)

sdf auth status

Show status of credentials / tokens

Usage: sdf auth status [OPTIONS] [TARGETS]...

Arguments:
  • TARGETS — Compile only the given source dirs, files or tables [default: all models]
Options:
  • --credentials-dir CREDENTIALS_DIR — Path to the file where credentials will be stored (default is platform specific)

sdf man

Display reference material, like the CLI, dialect specific functions, schemas for authoring and interchange

Usage: sdf man [OPTIONS] COMMAND

Subcommands:
  • cli — Display SDF’s command line interface
  • functions — Display SDF’s functions definitions
  • definition-schema — Display SDF’s yml blocks as a json schema doc [only: json]
  • event-schema — Display SDF’s trace events as a json schema only [only: json]
  • information-schema — Display SDF’s information schemas [only: sql]
  • error-codes
Options:
  • -s, --show SHOW — Display messages [default: progress]

    Default value: progress

    Possible values: progress, none

sdf man cli

Display SDF’s command line interface

Usage: sdf man cli [OPTIONS]

Options:
  • --format FORMAT — Format reference material in this format [only: markdown]

    Possible values: sql, yml, markdown, json

sdf man functions

Display SDF’s functions definitions

Usage: sdf man functions [OPTIONS]

Options:
  • --dialect DIALECT — Dialect for all functions

    Default value: trino

    Possible values: snowflake, trino, bigquery, redshift, spark-lp

  • --section SECTION — Section to display [ section value must appear in the function registry]

    Default value: all

  • --format FORMAT — Format reference material in this format [default: markdown | yml]

    Possible values: sql, yml, markdown, json

  • --implemented — limit output to only implemented functions

    Default value: false

    Possible values: true, false

sdf man definition-schema

Display SDF’s yml blocks as a json schema doc [only: json]

Usage: sdf man definition-schema [OPTIONS]

Options:
  • --format FORMAT — Format reference material in this format [only: json]

    Possible values: sql, yml, markdown, json

sdf man event-schema

Display SDF’s trace events as a json schema only [only: json]

Usage: sdf man event-schema [OPTIONS]

Options:
  • --format FORMAT — Format reference material in this format [only: json]

    Possible values: sql, yml, markdown, json

sdf man information-schema

Display SDF’s information schemas [only: sql]

Usage: sdf man information-schema [OPTIONS]

Options:
  • --format FORMAT — Format reference material in this format [only: sql]

    Possible values: sql, yml, markdown, json

sdf man error-codes

Usage: sdf man error-codes [OPTIONS]

Options:
  • --format FORMAT — Format reference material in this format [only: markdown]

    Possible values: sql, yml, markdown, json

sdf dbt

Initialize an sdf workspace from an existing dbt project

Usage: sdf dbt [OPTIONS] COMMAND

Subcommands:
  • init — Initialize a sdf workspace from a dbt project — best effort
  • refresh — Re-initialize a sdf workspace from a dbt project — best effort
Options:
  • --target-dir TARGET_DIR — Set the target-dir for this session

  • --credentials-dir CREDENTIALS_DIR — Read the credentials from credentials-dir

  • -s, --show SHOW — Display messages

    Default value: progress

    Possible values: progress, none

sdf dbt init

Initialize a sdf workspace from a dbt project — best effort

Usage: sdf dbt init [OPTIONS]

Options:
  • --target TARGET — Use this DBT target over the default target in profiles.yml

  • --profiles-dir PROFILES_DIR — Use this DBT profile instead of the defaults at ~/.dbt/profile.yml — (note dbt uses —profile_dir, this CLI uses —profile-dir)

  • --workspace-dir WORKSPACE_DIR — Specifies the workspace directory where we expect to see manifest and dbt project files The SDF workspace file will be placed in the same directory. Default: current directory

  • -n, --no-save — DEPRECATED: this option has no effect; see —save

    Default value: false

    Possible values: true, false

  • -s, --save — Save and overwrite the workspace file

    Default value: false

    Possible values: true, false

  • -c, --config CONFIG — Supply a config yml file or provide config as yml string e.g. ‘(key: value)’

sdf dbt refresh

Re-initialize a sdf workspace from a dbt project — best effort

Usage: sdf dbt refresh [OPTIONS]

Options:
  • --target TARGET — Use this DBT target over the default target in profiles.yml

  • --profiles-dir PROFILES_DIR — Use this DBT profile instead of the defaults at ~/.dbt/profile.yml — (note dbt uses —profile_dir, this CLI uses —profile-dir)

  • --workspace-dir WORKSPACE_DIR — Specifies the workspace directory where we expect to see manifest and dbt project files The SDF workspace file will be placed in the same directory. Default: current directory

  • -n, --no-save — DEPRECATED: this option has no effect; see —save

    Default value: false

    Possible values: true, false

  • -s, --save — Save and overwrite the workspace file

    Default value: false

    Possible values: true, false

  • -c, --config CONFIG — Supply a config yml file or provide config as yml string e.g. ‘(key: value)’

sdf exec

Execute custom scripts

Usage: sdf exec [OPTIONS] COMMAND

Arguments:
  • COMMAND — The command to be executed
Options:
  • -s, --show SHOW — The verbosity of the output

    Default value: all

    Possible values: all, progress, result, none

  • --path PATH — Execute command at the path given [default: current workspace directory]

    Finished man in 0.308 secs