martin o leary
07/18/2023, 10:23 AMrun_queue:
max_concurrent_runs: 6
tag_concurrency_limits:
- key: "dagster/backfill"
limit: 4
I'm wondering if Docker might be constraining what resources are available to dagit maybe?
Anyone seen anything like this before?martin o leary
07/18/2023, 10:27 AMmartin o leary
07/18/2023, 10:49 AMname : dagster
version : 1.3.14
description : The data orchestration platform built for productivity.
martin o leary
07/18/2023, 11:07 AMtag_concurrency_limits
key from my run_quue
config and it seems to have worked nowclaire
07/18/2023, 7:16 PMdagster/backfill
tag is applied to each run that is created via a backfill. So setting a limit of 4
there would enforce that a maximum of 4 runs triggered via a backfill can be in progress at any given time.martin o leary
07/18/2023, 7:49 PMJulien DEBLANDER
08/16/2023, 4:36 PMrun_queue:
max_concurrent_runs: 3
tag_concurrency_limits:
- key: "dagster/backfill"
limit: 3
- key: "dbt"
value: "incremental"
limit: 1
While defining the asset job I write the tag:
tags = {"dbt": "incremental"}
But still, backfills run 3 by 3.
Any thoughts on this? Thanks