Work with Variables in Jinja to parameterize your SQL and YML files
workspace.sdf.yml
file, can also be accessed in the same way as local variables. They are namespaced globally, and can be accessed in any file in the workspace.
<workspace-name>.<variable-name>
.
Let’s look at an example, imagine we have several models that all use the same list of countries. We can define a namespaced variable to store the list of countries, and use it in our SQL queries.
For the sake of the this example, let’s assume we have a workspace named transformations
. We can define a namespaced variable in a file named countries.jinja
as follows:
env_var(VAR)
. This function is accessible in both SQL, and YML files.
Navigate to Your Workspace
Turn on the macro preprocessor
workspace.sdf.yml
, in the workspace, block,
add the preprocessor configuration.Give it a Try!
Navigate to Your Workspace
Turn on the macro preprocessor
workspace.sdf.yml
, in the workspace, block,
add the preprocessor configuration.Give it a Try!
sdf compile
with the existing configuration. Notice that the catalog name of all
assets should be my_catalog
as we set it up in the previous step.Now, let’s introduce an environment variable.Change the default catalog to point to an environment variable. In this example, we will use the local
username:sdf compile
, the catalog name of all aassets will be {your_username}_catalog
, when {your_username}
is
the username you use for your local machine.printenv
in your terminal to view all available environment variables on your machine.Create a New Environment variable
DEFAULT_CATALOG
. For the sake of this example,
we can set it to be my_default_catalog
.In your terminal, run:env_var
reference we set in the previous step:sdf compile
again. This time, the catalog name of all aassets will match the new environment variable we set up -
my_default_catalog
..sdf.yml
files so that they are not stored in plain text.
Here’s an example of using these to hydrate Snowflake credentials in your project. Let’s start by creating a file creds.sdf.yml
and placing it next to your workspace.sdf.yml
file.
Next, add the following content to the file:
workspace.sdf.yml
like so: