Looking for some help with setting environment variables with Dagster and dbt. I have followed the layout of the "fully featured project" that is available as an example in GitHub. I have it working correctly when the database password is in plain text in the profiles.yml file. However, when I load my environment variables in Dagster as a .env file, they do not persist to dbt. So password in plain text runs fine, password referencing an environment variable fails. Any help would be appreciated! This is the last step to be able to set up our production code pipeline and run Dagster in production. TIA! Truncated folder setup:

.
└── workspace/
    ├── .env
    ├── dagster/
    │   ├── __init__.py
    │   ├── assets
    │   └── dbt_run.py
    └── dbt/
        ├── dbt_project.yml
        └── config/
            └── profiles.yml

Contents of .env (sanitized):

DAGSTER_DEPLOYMENT=local
DATABASE_PASSWORD=<my_password>
DATABASE_ROLE=<my_role>
DATABASE_SCHEMA=<my_schema>
DATABASE_USER=<my_user>

profiles.yml contents:

company:
  outputs:
    dev:
      account: <my_snowflake_account>
      database: <my_database>
      password: "{{ env_var('DBT_DATABASE_PASSWORD') }}" <-- this works when plain text, does not work when set as an env_var
      role: <my_role>
      schema: <my_schema>
      threads: 4
      type: snowflake
      user: <my_user>
      warehouse: <my_warehouse>

__init__.py to set up dbt (from the example, again, this works when the password is in plain text, does not work when I try to use the environment variable)
from dagster_dbt import load_assets_from_dbt_project
from dagster import file_relative_path
import os

DBT_DATABASE_PASSWORD = os.getenv("DATABASE_PASSWORD") <-- my attempt to load the env_var in Dagster first, this didn't work
DBT_PROJECT_PATH = file_relative_path(__file__, "../../dbt/")
DBT_PROFILES = file_relative_path(__file__, "../../dbt/config")

dbt_assets = load_assets_from_dbt_project(
    project_dir=DBT_PROJECT_PATH, profiles_dir=DBT_PROFILES, key_prefix=["company"]
)

hi @Lisa Cloutier! Is there any particular reason that you want the dbt project to grab the environment variable from

DBT_DATABASE_PASSWORD

, but have it set as

DATABASE_PASSWORD

in your

.env

file? I believe if you updated your profiles.yml to have

password: "{{ env_var('DATABASE_PASSWORD') }}"

then things would start working. If you do want to stick with the two separate names approach, you'll want to do something like this in your init file:

# copy the value of DATABASE_PASSWORD to DBT_DATABASE_PASSWORD
os.environ["DBT_DATABASE_PASSWORD"] = os.getenv("DATABASE_PASSWORD")

Ah, sorry if I wasn't clear, I tried it both ways, first with just "{{ env_var('DATABASE_PASSWORD') }}" which did not work. Then I tried setting the environment variable with the DBT_DATABASE_PASSWORD method, which also didn't work.

got it -- are you able to tell if the dagster process is getting the "DATABASE_PASSWORD" env var set properly? i.e. if you print(os.getenv("DATABASE_PASSWORD")) inside your dagster code, does the password show up?

Hi owen, thanks for replying! • Where would I look for the output of the print statement print(os.getenv("DATABASE_PASSWORD")) • Yes dagster dev is run from the workspace folder. It runs correctly when the password is in plain text in the profiles.yml file.

you'd see that print output in the same terminal where you ran

dagster dev

(that spits out a lot of output, so you might want to make it more obvious with some

print("-"*200)

nearby or something)