Hello - I am using Dagster 1.17 and creating asset...
# random
Hello - I am using Dagster 1.17 and creating assets using an asset factory approach. I need to limit the number of assets that are materialized at once. I have all assets being triggered by a reconciliation sensor and the assets are currently a one-to-one relationship with their upstream dependencies. I have set a run_coordinator in the dagster.yaml with tag_concurrency_limits to 1 and have tried all of the following to no avail: • Setting the
on the
function in my asset factory • Setting the
of the reconciliation sensor • Setting the tags in the config_schema of the
function In all instances, when the assets are deemed stale by the reconciliation sensor, a single job runs with all stale assets running concurrently. This risks putting too large of a strain on our upstream DB so we have to see this being limited. Can someone tell me what I might be missing here?