This guide covers advanced features and configurations for Snowflake integrations in SDF, like role configuration, case-preserving identifiers, and more.
SELECT TABLE | SELECT VIEW | CREATE <MATERIALIZATION> | CREATE SCHEMA | CREATE DATABASE | |
---|---|---|---|---|---|
compile | 🟢 | 🟢 | |||
run | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
CREATE MATERIALIZATION
permission listed above is meant to be subsituted with the materialization type you are using. For example, if you are using table
materializations, the permission should be CREATE TABLE
. If using views or transient tables, the permission should be CREATE VIEW
or CREATE TRANSIENT TABLE
respectively.
If you’d like to grant all write access on a database, these can be easily achieved with a grant all
command on the database.my_production_database
database and write models to the dev_sandbox
database.
my_production_database
being included as a target, SDF only requires write access to the databases and schemas after the rename operation. As such, the dev_sandbox
schema should be the only schema that requires write access.sdf_dev_role
with permissions to write to the dev_sandbox
database and read from the my_production_database
database. This role will model exactly what’s required to compile and run with the integration above.
to_upper
dialect for all models in Snowflae, so that you’re local model specification matches Snowflake’s behavior in the cloud. Rarely if not ever should identifiers have their case preserved from the file system. Doing so likely creates unnecessary confusion in how models will be materialized and how they should be referenced in Snowflake.CREATE TABLE my_model AS ...
in Snowflake, the newly created table will be called MY_MODEL
.
Snowflake, however, will preserve the case of identifiers if they are enclosed in double quotes. This means the following SQL would produce a table called MyModel
:
casing-policy
option to preserve
instead of to_upper
for that model.
casing-policy
property. Since Snowflake post-normalization differs in its behavior from Snowflake pre-normalization, we introduced a casing-policy
property to capture that behavior. In most dialects, this defaults to preserve
. However, Snowflake is the special child! Since Snowflake normalizes to upper case, we default the casing-policy
to to-upper
for Snowflake. You should only overwrite this if you need to preserve identifier casing, or handle SQL filenames that begin with numbers.casing-policy
property can be set at an individual table-block level, like so:
warehouse
property to the table-block of an sdf.yml
file.
The warehouse can be specified easily on the model level with the simple top-level warehouse specification like so:
incremental-options
and snapshot-options
configs respectively. Here’s an example:
BIG_WH
will be used by default for the first run, full refresh runs, and all tests, since tests will scan the history of all increments. Then, due to the compact-mode-warehouse
property, SMALL_WH
will be used for incremental runs.