I’m still finding the docs really convoluted and f...
# dagster-feedback
I’m still finding the docs really convoluted and frustrating. I’m trying to set up a dbt-cloud integration and have been following along with this page. A few issues came up: 1.
didn’t work for me. It didn’t seem to substitute the values that the var held. When I changed them to
, it worked fine. So, what are the missing pieces here to get
working in this context? 2. A job called
is defined that selects only assets in the
group. However, it never discusses how to add dbt assets to this group. When I tried it in my project, this job was empty. 3. Step 4 is optional, but it never mentions why you might want to use this option. It wasn’t until I was developing locally and noticed that dagster was spinning up a
dbt compile
job every few minutes that I realized that it’s practically necessary for local development.
Hey man I’m on the same journey as you are, I asked the same questions too. Maybe the responses I got from the team will be relevant for you as well. 1. On EnvVar 2. On dbt compile
Thanks Lydia. Any luck figuring out how to tell dagster to only run dbt models with certain tags or model paths?
For adding to the group,
which you can use with a simple lambda
node_info_to_group_fn=lambda node_info: "my_group"
to set them all as a static group (Feedback to anyone from dagster team: might be more ergonomic to allow for a static string version of the argument!)
It also includes a
argument you can use as you would the dbt command line
to import only certain tags (e.g.
) with that function, so you can select certain dbt assets to be loaded into one group, and others into another group.
I think dbt suggests using project path rather than tag for this sort of thing, but I've yet to move to that approach.
Thanks Brendan. I haven’t looked into it yet, but Owen told me that tag selectors only work with dbt-core. Have you done this with dbt-cloud?
Ah - I totally misread that.
That seems a big missing feature. 😕
Yeah, starting to think that I might need to finally learn how to use git submodules blob worry
😓 1
Hey @Sterling Paramore we were able to get the integration working but theres a bunch of limitations w/ dbt-cloud specifically i have not for the life of me figured out how to tell dagster to just run specific model paths using dbt-cloud without causing a lot of queueing problems.