https://dagster.io/ logo
#dagster-feedback
Title
# dagster-feedback
c

Charlie Bini

04/28/2023, 3:55 PM
Idea for AutoMaterialization: include a run tags param when setting a policy. It will be helpful for when you need to enforce
tag_concurrency_limits
1
For example, I have a partitioned dbt asset, and BigQuery enforces quota of 20 queued DML statements. If I have many partitions to run, they need to be run in batches of at most 20 or else they're start to fail
z

Zachary Bluhm

04/28/2023, 4:00 PM
Haha we just ran into that DML issue a couple weeks ago... Hoping op level concurrency will be our silver bullet: https://github.com/dagster-io/dagster/pull/13769#issuecomment-1515136631a