https://dagster.io/ logo
Title
f

Francis Niu

10/20/2021, 9:35 AM
Hi team, I’m using dagster-dbt, How can I pass environment variables into
dbt_cli_resource
so that they can be referenced in dbt profiles? like this: https://github.com/dagster-io/dagster/blob/master/examples/dbt_example/dbt_example_project/profiles.yml
c

chris

10/20/2021, 1:56 PM
Cc @owen
o

owen

10/20/2021, 3:37 PM
hi @Francis Niu -- the
dbt_cli_resource
is just responsible for launching commands against a specific dbt project. The variables inside a dbt profile (things like database credentials) generally do not interact with the variables that are configured in the dbt_cli_resource (things like project directory or profile name). How/where are you launching these runs? Generally, you'd want to have these environment variables set on whatever machine(s) you're running your dagster code on, rather than explicitly hard-coding these variables into your python code. If you want to be able to configure your dbt project differently depending on the run, you can always create multiple targets in your profiles.yml file, and pass a different target argument to dbt_cli_resource depending on your needs. As a last resort, you can always set environment variables explicitly in python code using
os.environ["SOME_VARIABLE"] = "some_value"
, but generally you shouldn't use this pattern unless you're just developing locally or writing tests.