> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sdf.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Logging

> SDF provides several options for configuring log output. Furthermore, the SDF DB can be used to query and analyze logs.

SDF outputs valuable context to `stdout` by default, but rich trace logs are also available for debugging and monitoring.

The following guide will go through the different logging options available in SDF and how to use them. See the [reference](#reference) for details on log formats, levels, and properties.

## Guide

<Steps>
  <Step title="Open an SDF workspace">
    Let's use our `moms_flower_shop_completed` sample workspace as a base for this guide. If you don't have it, you can create it by running:

    ```shell theme={null}
    sdf new --sample moms_flower_shop_completed && cd moms_flower_shop_completed
    ```
  </Step>

  <Step title="Explore Log Levels">
    By default, SDF logs progress, warnings, and errors.
    We can filter this down by specifying `--log-level warn` or `--log-level error` to only see warnings or errors, respectively.

    To see more in-depth logs, we can use the `--log-level info` flag. This will expose logs that capture the internal steps that sdf follows to compile and run a workspace.

    Let's say we want to view info logs along with error and warn logs, we can run:

    ```shell theme={null}
    sdf compile moms_flower_shop.raw.raw_inapp_events --log-level info --show none
    ```

    <div className="bg-[#0F1117] dark:bg-codeblock rounded-xl dark:ring-1 dark:ring-gray-800/50 relative">
      <pre style={{ fontFamily: 'monospace', backgroundColor: 'transparent' }} className="language-shell">
        <code className="language-shell">
          \{"\_ts":"2025-02-05T02:02:12.455233347+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/main.rs:105","ev":"sdf"}
          \{"\_ts":"2025-02-05T02:02:12.455314929+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/main.rs:106","ev":"sdf","set":"job\_id","val\_s":"64b1ab20-5af0-44b8-948c-599ed9bb9c19"}
          \{"\_ts":"2025-02-05T02:02:12.455325619+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/main.rs:107","ev":"sdf","set":"args","val\_as":\["compile","moms\_flower\_shop.raw\.raw\_inapp\_events","--log-level","info","--show","none"]}
          \{"\_ts":"2025-02-05T02:02:12.456481748+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:120","ev":"initialize"}
          \{"\_ts":"2025-02-05T02:02:12.457853532+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:300","ev":"initialize.load\_workspace"}
          \{"\_ts":"2025-02-05T02:02:12.477270421+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:369","ev":"initialize.load\_workspace","st\_dur\_ms":19}
          \{"\_ts":"2025-02-05T02:02:12.477287553+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:371","ev":"initialize.load\_dependencies"}
          \{"\_ts":"2025-02-05T02:02:12.477303142+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:502","ev":"initialize.load\_dependencies","st\_dur\_ms":0}
          \{"\_ts":"2025-02-05T02:02:12.477315686+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:504","ev":"initialize.load\_prelude"}
          \{"\_ts":"2025-02-05T02:02:12.480955030+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:540","ev":"initialize.load\_prelude","st\_dur\_ms":3}
        </code>
      </pre>
    </div>

    <Note>
      We used `--show none` to avoid displaying stdout output. The `--show` flag controls the standard out, things like "compiling \<table>", and is separate from info, debug, and trace logs.
    </Note>

    Nice! We've lifted up the hood on SDF and are seeing info logs in action.

    `debug` and `trace` logs are also available, and are more detailed than `info` logs. Let's try trace, the most detailed log level:

    ```shell theme={null}
    sdf compile moms_flower_shop.raw.raw_inapp_events --log-level trace --show none
    ```

    <div className="bg-[#0F1117] dark:bg-codeblock rounded-xl dark:ring-1 dark:ring-gray-800/50 relative">
      <pre style={{ fontFamily: 'monospace', backgroundColor: 'transparent' }} className="language-shell">
        <code className="language-shell">
          \{"\_ts":"2025-02-05T02:02:14.547085928+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:579","ev":"initialize.prep\_env.resolve\_env.build\_env","st\_dur\_ms":603}
          \{"\_ts":"2025-02-05T02:02:14.547161469+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_test at sdftarget/dbg/.lib/sdf\_test/macros/test.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547424460+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_utils at sdftarget/dbg/.lib/sdf\_utils/macros/generate\_surrogate\_key.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547469084+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_utils at sdftarget/dbg/.lib/sdf\_utils/macros/group\_by.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547505692+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_utils at sdftarget/dbg/.lib/sdf\_utils/macros/generate\_date\_values.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547559803+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_utils at sdftarget/dbg/.lib/sdf\_utils/macros/date\_spine.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547629042+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_utils at sdftarget/dbg/.lib/sdf\_utils/macros/generate\_integer\_values.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.547665360+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/context/env.rs:900","message":"Reading jinja file: sdf\_mat at sdftarget/dbg/.lib/sdf\_mat/macros/dialect/trino/materialize.jinja"}
          \{"\_ts":"2025-02-05T02:02:14.548078591+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/utils/initializing.rs:755","ev":"initialize.prep\_env.resolve\_env","st\_dur\_ms":604}

          \{"\_ts":"2025-02-05T02:02:14.548851424+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:521","ev":"cmd.prepare.read\_makefile","set":"verdict","val\_s":"full-rebuild"}
          \{"\_ts":"2025-02-05T02:02:14.548862224+00:00","\_ll":"DEBUG","\_fn":"crates/sdf-cli/src/subcommands/build.rs:577","ev":"cmd.prepare.read\_makefile","set":"analyze\_with\_sources","val\_b":false}
          \{"\_ts":"2025-02-05T02:02:14.548875930+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:582","ev":"cmd.prepare.read\_makefile","set":"resolved\_table\_names","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.548882723+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:588","ev":"cmd.prepare.read\_makefile","set":"seen\_these\_files","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.548889225+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:594","ev":"cmd.prepare.read\_makefile","set":"sources","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.548895486+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:605","ev":"cmd.prepare.read\_makefile","st\_dur\_ms":0}

          \{"\_ts":"2025-02-05T02:02:14.556741410+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1654","ev":"cmd.schedule.prep"}
          \{"\_ts":"2025-02-05T02:02:14.556776987+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1677","ev":"cmd.schedule","set":"deps","val\_as":\{"moms\_flower\_shop.raw\.raw\_inapp\_events":\[]}}
          \{"\_ts":"2025-02-05T02:02:14.556817162+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1757","ev":"cmd.schedule.prep","st\_dur\_ms":0}
          \{"\_ts":"2025-02-05T02:02:14.556826329+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1854","ev":"cmd.schedule","set":"resolved\_tables","val\_as":\{}}
          \{"\_ts":"2025-02-05T02:02:14.556835717+00:00","\_ll":"DEBUG","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1860","ev":"cmd.schedule","set":"sources\_defined\_by\_providers","val\_as":\[]}
          \{"\_ts":"2025-02-05T02:02:14.556843050+00:00","\_ll":"DEBUG","\_fn":"crates/sdf-cli/src/subcommands/build.rs:1906","ev":"cmd.schedule","set":"to\_be\_downloaded","val\_as":\[]}
          \{"\_ts":"2025-02-05T02:02:14.556891541+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2079","ev":"cmd.schedule.dag"}
          \{"\_ts":"2025-02-05T02:02:14.556901710+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2081","ev":"cmd.schedule.dag","set":"dag.deps","val\_s":"\{\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;:\[]}"}
          \{"\_ts":"2025-02-05T02:02:14.556917539+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2095","ev":"cmd.schedule.dag","set":"dag.initial","val\_s":"\{\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;:\[]}"}
          \{"\_ts":"2025-02-05T02:02:14.556926987+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2107","ev":"cmd.schedule.dag","set":"dag.cleaned","val\_s":"\{\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;:\[]}"}
          \{"\_ts":"2025-02-05T02:02:14.556935453+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2113","ev":"cmd.schedule.dag","set":"missing\_defs","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.556954147+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2191","ev":"cmd.schedule.dag","set":"dag.resolved\_targets","val\_s":"\[\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;]"}
          \{"\_ts":"2025-02-05T02:02:14.556968384+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2219","ev":"cmd.schedule.dag","set":"dag.targeted","val\_s":"\{\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;:\[]}"}
          \{"\_ts":"2025-02-05T02:02:14.557006616+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2278","ev":"cmd.schedule.dag","set":"dag.acyclic","val\_s":"\{\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;:\[]}"}
          \{"\_ts":"2025-02-05T02:02:14.557014821+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2284","ev":"cmd.schedule.dag","set":"cycles","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.557020862+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2290","ev":"cmd.schedule.dag","set":"cut\_points","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.557031783+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2296","ev":"cmd.schedule.dag","set":"order.initial","val\_s":"\[\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;]"}
          \{"\_ts":"2025-02-05T02:02:14.557063341+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2565","ev":"cmd.schedule.dag","set":"showable\_root\_tables","val\_s":"\[]"}
          \{"\_ts":"2025-02-05T02:02:14.557082868+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2854","ev":"cmd.schedule.dag","set":"order.without\_providers","val\_s":"\[\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;]"}
          \{"\_ts":"2025-02-05T02:02:14.557091324+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2880","ev":"cmd.schedule.dag","st\_dur\_ms":0}

          \{"\_ts":"2025-02-05T02:02:14.557105821+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2887","ev":"cmd.do"}
          \{"\_ts":"2025-02-05T02:02:14.557115589+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2913","ev":"cmd.do","set":"order.without\_providers\_or\_cached","val\_s":"\[\&quot;moms\_flower\_shop.raw\.raw\_inapp\_events\&quot;]"}
          \{"\_ts":"2025-02-05T02:02:14.557126529+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:2930","ev":"cmd.do.sources"}

          \{"\_ts":"2025-02-05T02:02:14.557863656+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:4342","ev":"cmd.do.derived.wave\_1.mat","ev\_tb":"moms\_flower\_shop.raw\.raw\_inapp\_events","ev\_tb\_catalog":"moms\_flower\_shop","ev\_tb\_schema":"raw","ev\_tb\_table":"raw\_inapp\_events"}
          \{"\_ts":"2025-02-05T02:02:14.558782872+00:00","\_ll":"DEBUG","\_fn":"crates/sdf-cli/src/subcommands/describe.rs:261","ev":"cmd.do.derived.wave\_1.meta\_analysis","ev\_tb":"moms\_flower\_shop.raw\.raw\_inapp\_events","ev\_tb\_catalog":"moms\_flower\_shop","ev\_tb\_schema":"raw","ev\_tb\_table":"raw\_inapp\_events"}
          \{"\_ts":"2025-02-05T02:02:14.558801737+00:00","\_ll":"DEBUG","\_fn":"crates/sdf-cli/src/subcommands/describe.rs:284","ev":"cmd.do.derived.wave\_1.meta\_analysis","ev\_tb":"moms\_flower\_shop.raw\.raw\_inapp\_events","ev\_tb\_catalog":"moms\_flower\_shop","ev\_tb\_schema":"raw","ev\_tb\_table":"raw\_inapp\_events","st\_dur\_ms":0}
          \{"\_ts":"2025-02-05T02:02:14.558859575+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:4466","ev":"cmd.do.derived.wave\_1.mat","ev\_tb":"moms\_flower\_shop.raw\.raw\_inapp\_events","ev\_tb\_catalog":"moms\_flower\_shop","ev\_tb\_purpose":"model","ev\_tb\_schema":"raw","ev\_tb\_table":"raw\_inapp\_events","st\_code":"succeeded","st\_done":"executed","st\_dur\_ms":0}

          \{"\_ts":"2025-02-05T02:02:14.558913185+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:4326","ev":"cmd.do.derived","st\_dur\_ms":1}
          \{"\_ts":"2025-02-05T02:02:14.559349793+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/utils/dep\_cache.rs:1890","message":"Saving table cache: moms\_flower\_shop.raw\.raw\_inapp\_events, sdftarget/dbg/table/moms\_flower\_shop/raw/raw\_inapp\_events.sdf.yml"}
          \{"\_ts":"2025-02-05T02:02:14.559761539+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:3509","ev":"cmd.do.gen\_makefile"}
          \{"\_ts":"2025-02-05T02:02:14.560011345+00:00","\_ll":"TRACE","\_fn":"crates/sdf-cli/src/utils/dep\_cache.rs:2300","ev":"written\_makefile","set":"makefile","val\_m":\{"contents":\{"cli\_vars":"99914b932bd37a50b983c5e7c90ae93b","credentials":"99914b932bd37a50b983c5e7c90ae93b","env\_vars":"d751713988987e9331980363e24189ce"},"prelude":\[\{"file":"workspace.sdf.yml","mtime":"2025-02-05T02:02:12.119151972Z"},\{"file":"metadata/analytics/dim\_marketing\_campaigns.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/raw/raw\_customers.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/raw/raw\_marketing\_campaign\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/analytics/agg\_installs\_and\_campaigns.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/raw/raw\_addresses.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/staging/inapp\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/raw/raw\_inapp\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/staging/app\_installs.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"classifications/column\_classifiers.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"classifications/table\_classifiers.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/analytics/dim\_marketing\_campaigns.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/raw/raw\_customers.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/raw/raw\_marketing\_campaign\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/analytics/agg\_installs\_and\_campaigns.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/raw/raw\_addresses.sdf.yml","mtime":"2025-02-05T02:02:12.117151970Z"},\{"file":"metadata/staging/inapp\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/raw/raw\_inapp\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"metadata/staging/app\_installs.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"sdftarget/dbg/.lib/sdf\_test/workspace.sdf.yml","mtime":"2025-02-05T02:02:13.939153370Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/workspace.sdf.yml","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_mat/workspace.sdf.yml","mtime":"2025-02-05T02:02:13.939153370Z"},\{"file":"sdftarget/dbg/.lib/sdf\_test/macros/test.jinja","mtime":"2025-02-05T02:02:13.939153370Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/macros/generate\_surrogate\_key.jinja","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/macros/group\_by.jinja","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/macros/generate\_date\_values.jinja","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/macros/date\_spine.jinja","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_utils/macros/generate\_integer\_values.jinja","mtime":"2025-02-05T02:02:13.940153371Z"},\{"file":"sdftarget/dbg/.lib/sdf\_mat/macros/dialect/trino/materialize.jinja","mtime":"2025-02-05T02:02:13.939153370Z"}],"deps":\[\{"target":"moms\_flower\_shop.raw\.raw\_inapp\_events","deps":\[]}],"source\_files":\{"values":\{"moms\_flower\_shop.raw\.raw\_inapp\_events":\[\{"file":"metadata/raw/raw\_inapp\_events.sdf.yml","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"models/raw/raw\_inapp\_events.sql","mtime":"2025-02-05T02:02:12.118151971Z"},\{"file":"seeds/parquet/inapp\_events.parquet","mtime":"2025-02-05T02:02:12.119151972Z"}]}},"incomplete\_source\_files":\{"values":\[]},"target\_status":\{"values":\{"moms\_flower\_shop.raw\.raw\_inapp\_events":\{"purpose":"model","status":"succeeded","draft":false}}}}}
          \{"\_ts":"2025-02-05T02:02:14.560186646+00:00","\_ll":"INFO","\_fn":"crates/sdf-cli/src/subcommands/build.rs:3523","ev":"cmd.do.gen\_makefile","st\_dur\_ms":0}
        </code>
      </pre>
    </div>

    Notice that the trace logs include `info`, `debug`, and `trace` logs. Furthermore, the `trace` log is *very* *very* long. Trace logs are typically used for internal use cases, such as debugging SDF itself. As such, we recommend using `info` or `debug` logs for most use cases.
  </Step>

  <Step title="Writing Logs to Files">
    By default, SDF logs to `stdout`. However, we can write logs to a file by specifying the `--log-file` flag.

    Let's write the logs to a file named `sdf.log`:

    ```shell theme={null}
    sdf compile moms_flower_shop.raw.raw_inapp_events --log-level info --log-file sdf.log --show none
    ```

    This command will produce no output, but if we inspect the new file `sdf.log`, we'll see the logs output to the file.

    The logs outputted to the file respect the `--log-level` flag you've set.
  </Step>

  <Step title="Bonus: Analyze the Logs with SDF DB">
    As mentioned earlier, SDF's in-memory database can be used to query and analyze logs, right from the terminal on your machine.

    Let's start by gathering up all the info logs from a fresh compile.

    First, let's clean the workspace cache:

    ```shell theme={null}
    sdf clean
    ```

    Now, let's compile the workspace and output the logs to a file called `logs/compile_info.json`:

    ```shell theme={null}
    sdf compile moms_flower_shop.raw.raw_inapp_events --log-level info --log-file logs/compile_info.json --show none
    ```

    Now that we have the logs written to a file, let's include them in our SDF workspace by adding the following to the `includes` block in your `workspace.sdf.yml`:

    ```yml theme={null}
    workspace:
    ...
      includes:
        - path: logs
          type: seed
    ```

    Finally, let's compile our new logs model to ensure SDF can process it:

    ```shell theme={null}
    sdf compile moms_flower_shop.pub.compile_info
    ```

    <div className="bg-[#0F1117] dark:bg-codeblock rounded-xl dark:ring-1 dark:ring-gray-800/50 relative">
      <pre style={{ fontFamily: 'monospace', backgroundColor: 'transparent' }} className="language-shell">
        <code className="language-shell">
          Working set 12 model files, 1 seed file, 19 .sdf files
             Finished 1 model \[1 succeeded] in 1.102 secs

          Schema moms\_flower\_shop.pub.compile\_info
          ┌───────────────┬────────────────┬────────────┬─────────────┐
          │ column\_name   ┆ data\_type      ┆ classifier ┆ description │
          ╞═══════════════╪════════════════╪════════════╪═════════════╡
          │ \_ts           ┆ varchar        ┆            ┆             │
          │ \_ll           ┆ varchar        ┆            ┆             │
          │ \_fn           ┆ varchar        ┆            ┆             │
          │ ev            ┆ varchar        ┆            ┆             │
          │ set           ┆ varchar        ┆            ┆             │
          │ val\_s         ┆ varchar        ┆            ┆             │
          │ val\_as        ┆ array\<varchar> ┆            ┆             │
          │ st\_dur\_ms     ┆ bigint         ┆            ┆             │
          │ ev\_fn         ┆ varchar        ┆            ┆             │
          │ st\_code       ┆ varchar        ┆            ┆             │
          │ ev\_tb         ┆ varchar        ┆            ┆             │
          │ ev\_tb\_catalog ┆ varchar        ┆            ┆             │
          │ ev\_tb\_schema  ┆ varchar        ┆            ┆             │
          │ ev\_tb\_table   ┆ varchar        ┆            ┆             │
          │ ev\_tb\_purpose ┆ varchar        ┆            ┆             │
          │ st\_done       ┆ varchar        ┆            ┆             │
          │ err\_msg       ┆ varchar        ┆            ┆             │
          │ exit\_code     ┆ bigint         ┆            ┆             │
          └───────────────┴────────────────┴────────────┴─────────────┘
        </code>
      </pre>
    </div>

    Awesome! SDF can process the logs model. Let's run a quick inline query to select some properties from the logs:

    ```shell theme={null}
    sdf run -q 'select "_ts" as timestamp, "ev" as event, "st_dur_ms" as duration_in_ms from compile_info'
    ```

    <div className="bg-[#0F1117] dark:bg-codeblock rounded-xl dark:ring-1 dark:ring-gray-800/50 relative">
      <pre style={{ fontFamily: 'monospace', backgroundColor: 'transparent' }} className="language-error">
        <code className="language-error">
          Working set 13 model files, 1 seed file, 19 .sdf files
              Running moms\_flower\_shop.pub.query (./sdftarget/dbg/query.sql)
             Finished 2 models \[2 succeeded] in 1.111 secs

          Table moms\_flower\_shop.pub.query
          ┌─────────────────────────────────────┬──────────────────────────────┬────────────────┐
          │ timestamp                           ┆ event                        ┆ duration\_in\_ms │
          ╞═════════════════════════════════════╪══════════════════════════════╪════════════════╡
          │ 2025-02-05T02:02:16.436983278+00:00 ┆ sdf                          ┆                │
          │ 2025-02-05T02:02:16.437122638+00:00 ┆ sdf                          ┆                │
          │ 2025-02-05T02:02:16.437196325+00:00 ┆ sdf                          ┆                │
          │ 2025-02-05T02:02:16.438328098+00:00 ┆ initialize                   ┆                │
          │ 2025-02-05T02:02:16.440028309+00:00 ┆ initialize.load\_workspace    ┆                │
          │ 2025-02-05T02:02:16.463394531+00:00 ┆ initialize.load\_workspace    ┆ 23             │
          │ 2025-02-05T02:02:16.463470382+00:00 ┆ initialize.load\_dependencies ┆                │
          │ 2025-02-05T02:02:16.463549049+00:00 ┆ initialize.load\_dependencies ┆ 0              │
          │ 2025-02-05T02:02:16.463612187+00:00 ┆ initialize.load\_prelude      ┆                │
          │ 2025-02-05T02:02:16.467722160+00:00 ┆ initialize.load\_prelude      ┆ 4              │
          └─────────────────────────────────────┴──────────────────────────────┴────────────────┘
          60 rows, showing only 10 rows.
            Run with --limit 0 to show all rows.
        </code>
      </pre>
    </div>

    Nice! We've successfully queried the logs using the SDF DB. Any models you create from the logs can be used to analyze and query the logs further.
    To learn more about the SDF DB, visit our [SDF DB Guide](/database/introduction).
  </Step>
</Steps>

## Reference

As a quick reference, let's review the different log levels, format, and properties within a log:

#### **Log Levels**

* `info` - general information about the execution of the command
* `debug` - more detailed information about the execution of the command
* `trace` - very detailed information about the execution of the command

#### **Log Properties**:

* `_ts` - the timestamp of the log
* `_lvl` - the log level (info, debug, trace, etc.)
* `id` - the ID of the variable being logged
* `val_s` - the value of a variable being logged if it's a string
* `val_a*` - the value of a variable being logged if it's an array, the `*` represents the value type of the array. I.e. `as` for an array of strings
* `val_m*` - the value of a variable being logged if it's a map, the `*` represents the value type of the map. I.e. `mss` for a map of string to string
* `val_b` - the value of a variable being logged if it's a boolean
* `val_i` - the value of a variable being logged if it's an integer
* `ith` - the index of the wave being logged
* `num_files` - the number of files being processed by an operation
* `file` - the filename being processed
* `num_errors` - the number of errors encountered during an operation
* `num_statements` - the number of statements being processed by an operation
* `num_tables` - the number of tables being processed by an operation
* `table` - the table being processed by an operation
* `message` - a message associated with the log
* `error_code` - the error code associated with the log (`0` if no error)
* `_src` - the source of the log in the Rust codebase
* `ev` - the event name corresponding to this log. Used to map `open` and `close` events
* `ev_type` - the type of the event (see reference below)
* `ev_dur_ms` - the duration of the event in milliseconds
* `status` - the status of the event (see reference below)

<Info>
  Why all the val\_ properties? SDF logs are designed to be queryable by engines that support structured types. As such, we map the Rust variable types to structured types like so:

  ```
  Rust types
  h - hashmap
  z - hashset
  a - array
  v - vector
  o - option
  t - table_name
  s - string
  n - number
  b - bool

  Json (also SQL) Types
  m - map
  a - array
  n - number
  b - bool
  s - string
  ```

  So, `msas` is map of string to array of string
</Info>

#### **Event Types**:

* `open` - the start of an event
* `close` - the end of an event. Use the `ev` property to map this to the corresponding `open` event
* `state` - a summary of state at a given point in time for an event
* `wave` - represents a batch operation operating on a subset of a dag or process. Waves have state, duration, and status properties

#### **Status**:

* `succeeded` - the event was successful
* `failed` - the event failed
* `skipped` - the event was skipped