Sterling Paramore
08/01/2023, 12:49 AMEnvVar
didn’t work for me. It didn’t seem to substitute the values that the var held. When I changed them to os.getenv
, it worked fine. So, what are the missing pieces here to get EnvVar
working in this context?
2. A job called run_staging_job
is defined that selects only assets in the "staging"
group. However, it never discusses how to add dbt assets to this group. When I tried it in my project, this job was empty.
3. Step 4 is optional, but it never mentions why you might want to use this option. It wasn’t until I was developing locally and noticed that dagster was spinning up a dbt compile
job every few minutes that I realized that it’s practically necessary for local development.Lydia Lim Li Han
08/01/2023, 4:15 AMSterling Paramore
08/01/2023, 5:04 PMBrendan Jackson
08/02/2023, 3:17 PMload_assets_from_dbt_project
accepts node_info_to_group_fn
which you can use with a simple lambda node_info_to_group_fn=lambda node_info: "my_group"
to set them all as a static group
(Feedback to anyone from dagster team: might be more ergonomic to allow for a static string version of the argument!)Brendan Jackson
08/02/2023, 3:18 PMselect
argument you can use as you would the dbt command line select
to import only certain tags (e.g. tag:my-tag
) with that function, so you can select certain dbt assets to be loaded into one group, and others into another group.Brendan Jackson
08/02/2023, 3:20 PMSterling Paramore
08/02/2023, 3:29 PMBrendan Jackson
08/02/2023, 3:30 PMBrendan Jackson
08/02/2023, 3:30 PMSterling Paramore
08/02/2023, 3:31 PMSon Do
08/04/2023, 2:58 PM