SDF Environments were built for teams to help them collaborate, isolate, and maintain development, staging, and production environments.
sandbox_
.
For sake of simplicity, we’ll use a local provider in this guide, but the same principles apply to remote providers.
Setup Moms Flower Shop
moms_flower_shop
sample.Add an Environment
dev
. This environment will use the local
provider to hydrate tables from the prod_db
database.
We’re also specifying that any tables materialized to the prod_db
database should, if run with this environment, materialize to the sandbox_prod_db
.Compile with the Environment
dev
environment, you can run:aka <renamed-table-name>
after each compiler output).Run with the Environment
dev
environment, you can run:(Optional) Test on a Remote Provider
dev
. This environment will use the snowflake
provider to hydrate tables from the prod_db
database.
We’re also specifying that any tables materialized to the prod_db
database should, if run with this environment, materialize to the sandbox_prod_db
.local_dev
credential to authenticate with Snowflake when sdf compile
or sdf run
is run with the dev
environment. This configuration allows for environments to have different credentials for different providers.rename-as
field is a Jinja template when used with *
. In this case, we’re renaming the
schema to sandbox_prod_db
and keeping the schema and table names the same using ${1}
and ${2}
respectively.
More about renaming in the next section.dev
environment. We can do this by using the following configuration:
*.*.*
is a wildcard pattern that matches any schema and table name.${1}
is the first match group from the source pattern. In this case, it’s the database name. Note that in the previous example, our first *
represented the schema name since it came after the database name like <database-name>.*
.sandbox_{{ env_var('developer_name') }}_${2}.${3}
is the new schema and table name. We’re prepending sandbox_
to the schema name, then using the env_var
function to get the developer’s name from the environment variables. Finally, we’re appending the table name to the end with ${3}
since that’s the third match group from the source pattern.env_var
jinja function for environments and other SDF yml use cases, a common patter is to use a .env
file to store environment variables. This way, you can keep sensitive information out of your codebase.dev
environment. You can do this by adding the following configuration:
catalog
, schema
, and materialization
configurations in the workspace. When running in the dev
environment, SDF will use the sandbox_catalog
catalog, the sandbox_<developer_name>
schema, and materialize tables as views
.