Hi guys, is there a way to set max_concurrency for...
# integration-dbt
y
Hi guys, is there a way to set max_concurrency for dbt runs in dagster?
🤖 1
r
y
Hi @rex, yes I looked at it, and not sure how to add tags for dbt assets, here's how we create dbt assets:
r
When you create a job using
define_asset_job
, you can supply the tags there: https://docs.dagster.io/_apidocs/assets#dagster.define_asset_job
y
Hi @rex, thanks for recommendation, however, we are trying to use auto materialization policy, and would like to allow only one dbt run per time, is there a way to add ops_tags like this:
r
@owen do we have a recommendation here?
o
ah interesting -- there's no way to directly set the op tags for a particular
@dbt_asset
, but all ops created with that decorator will automatically have the
{"kind": "dbt"}
tag set, which would probably do the trick for limiting concurrency in your case
y
nice! I will try that out! Thanks a lot!
Also is there a way to set up auto materialization that only materialize child asset when all of its parent assets are refreshed?
o
At the moment, no, but this is a feature several users have requested, and there's an open issue for it here: https://github.com/dagster-io/dagster/issues/14950
❤️ 1
y
Thanks a lot!
t
@owen, Following on from Yuan’s comment, what is the current recommended approach to handle this then? Just to ensure we are talking about the same problem, a small example where in the below flowchart, we want Asset C to only auto-materialize once Assets A and B have successfully materialized (e.g. Asset A is in Job A which is triggered by a sensor and Asset B is in Job B which runs on a cron schedule once per day). How would we handle this use case?
Copy code
Asset A -----|
             |------ Asset C
Asset B -----|
o
Hi! As of next week, this feature will be supported natively:
Copy code
wait_for_all_parents_policy = AutoMaterializePolicy.eager().with_rules(
    AutoMaterializeRule.skip_on_not_all_parents_updated()
)
🙌 1
t
Perfect timing Owen