Generate observability report

Elementary data observability report can be used for visualization and exploration of data from the dbt-package tables. That includes dbt test results, Elementary anomaly detection results, dbt artifacts, tests runs, etc.

Before you start

Before you can start using the report, make sure to install the dbt package, configure a profile and install the CLI. This is required for the report to work.

dbt package installation

How to install Elementary dbt package?

Configure connection profile

Configuring the Elementary Profile

In order to connect, Elementary needs a connection profile in a file named profiles.yml. This profile will be used by the CLI, to connect to the DWH and find the dbt package tables.

The easiest way to generate the profile is to run the following command within the dbt project where you deployed the elementary dbt package (works in dbt cloud as well):

dbt run-operation elementary.generate_elementary_cli_profile

Ensure that you fill in your password and any other missing fields after you paste the profile in your local profiles.yml file.

Copy the output, fill in the missing fields and add the profile to your profiles.yml. Here is a demonstration:

Elementary profile details and format

• Path: HOME_DIR/.dbt/profiles.yml
• Profile name: elementary
• Schema name: The schema of elementary models, default is <your_dbt_project_schema>_elementary

Snowflake

## SNOWFLAKE ##
## By default, edr expects the profile name 'elementary'.      ##
## Configure the database and schema of elementary models.     ##
## Check where 'elementary_test_results' is to find it.        ##

elementary:
  outputs:
    default:
      type: snowflake
      account: [account id]

      ## User/password auth ##
      user: [username]
      password: [password]

      role: [user role]
      database: [database name]
      warehouse: [warehouse name]
      schema: [schema name]_elementary
      threads: 4

Install CLI

Install Elementary CLI

To install the monitor module run:

pip install elementary-data

Run one of the following commands based on your platform (no need to run all):

pip install 'elementary-data[snowflake]'
pip install 'elementary-data[bigquery]'
pip install 'elementary-data[redshift]'
pip install 'elementary-data[databricks]'
pip install 'elementary-data[athena]'
## Postgres doesn't require this step

Run edr --help in order to ensure the installation was successful.

If you're receiving command not found: edr please check our troubleshooting guide.

Generate Tests Report UI

Execute edr report in your terminal

After installing and configuring the CLI, execute the command:

edr report

The command will use the provided connection profile to access the data warehouse, read from the Elementary tables, and generate the report as an HTML file.

---

Generating a report for single invocation

Elementary support filtering the report by invocation on generation. The filtered report will only include data for the selected invocation (This applies only on the Test Results and Lineage screens).

There are 3 ways to filter the report by invocation:

• Last invocation - Filter by the last invocation (dbt test / dbt build) that ran.
• Invocation time - Filter by the closest invocation (dbt test / dbt build) to the provided time (the provided time should be in ISO format local time).
• Invocation id - Filter by the provided invocation (dbt test / dbt build) id.

Filters usage example:

edr report invocation filters

edr report --select last_invocation
edr report --select "invocation_time:2022-12-25 10:10:35"
edr report --select invocation_id:XXXXXXXXXXXXX

---

Sharing the report

Elementary offers 3 methods that make it easy to share the report with others:

• Send via Slack
• Host on Amazon S3
• Host on Google Cloud Storage (GCS)

Refer to this guide for detailed instructions.