Skip to main content

​
Overview

Jinja is a powerful and popular templating engine. SDF uses Mini Jinja, a lightweight version of the Pythonic Jinja designed specifically for Rust applications. It provides similar functionality to Jinja but is tailored for integration with Rust projects, offering enhanced performance.
SDF comes with Jinja support built-in! No additional installations are required.

​
How Does it Work?

Jinja (or Mini Jinja) enables developers to embed logic and dynamic content into static templates. It leverages a unique syntax which resembles Python and is added directly to a file:
  • Jinja blocks should to be wrapped with {% and %}
  • Jinja variables should be wrapped with {{ and }}
When the file is invoked by the program, in this case SDF, the template gets resolved, changing the file dynamically based on the template.
This guide provides a high level overview of Jinja basics. We highly recommend to learn more from the official Jinja documentation and other sources online.

​
Common Templating in Data Pipelines

​
Variables

Let’s look at a simple example to make this more concrete. For simplicity, let’s assume we know table_name = orders:
In the code blocks below, toggle between the queries jinja templates and the rendered versions of the queries. The rendered versions of the queries are the ones that will be compiled and executed by SDF.
SELECT *
FROM {{ table_name }}
In order for this to work, we need to define the table_name Jinja variable. This is done using a set block: 
{% set table_name = 'orders' %}
SELECT *
FROM {{ table_name }}

​
If Statements

Let’s make it more interesting. If the environment name is ‘dev’, we want to query dev_orders, otherwise, orders. For that, we can use an if Jinja block. Notice, we need to close the if block with a endif block 
{% if builtin.environment == 'dev' %}  
    {% set table_name = 'dev_orders' %}
{% else %}
    {% set table_name = 'orders' %}
{% endif %}
SELECT *
FROM {{ table_name }}
SDF has many built-in Jinja variables that can be accessed in your models. Specifically, builtin.environment returns the environment name of the current environment.

​
For Loops

Jinja supports advanced data structures like lists and dictionaries. Notice that similarly to if statements, for loop blocks need to end with endfor blocks. 
{% set columns = ['order_id', 'customer_id', 'order_type'] %}
SELECT 
    {% for column_name in columns %}
        {{ column_name }},
    {% endfor %}
    COUNT(*) AS cnt
FROM orders
GROUP BY 
    {{ columns|join(', ') }}
Jinja supports some Python native methods. For example, the join method of lists as can be seen in the example above.
Now, let’s see how to iterate on a dictionary: 
{% set columns = {
    'id': 'order_id', 
    'uid': 'customer_id', 
    'type': 'order_type'
    }
 %}
SELECT 
    {% for column_name, column_rename in columns|items() %}
        {{ column_name }} AS {{ column_rename }},
    {% endfor %}
    ...
FROM orders
Another way to iterate on dictionaries is to iterate directly on its keys: 
{% set columns = {
    'id': 'order_id', 
    'uid': 'customer_id', 
    'type': 'order_type'
    }
 %}
SELECT 
    {% for column_name in columns %}
        {{ column_name }} AS {{ columns[column_rename] }},
    {% endfor %}
    ...
FROM orders
Mini Jinja syntax differs slightly from Pythonic Jinja, but it will still meets all your Jinja needs.For example, although iterating over dictionaries is slightly different, both libraries can implement the same logic with minor adjustments.

​
Resources

Jinja templating is incredibly powerful and there is much more to learn. We hope this guide helped provide an introduction to Jinja, but we encourage you to learn more about what you can do.
⌘I