https://dagster.io/ logo
#random
Title
q

Qumber Ali

11/12/2021, 12:42 PM
Hi all, I'm facing an issue regarding concurrency, here is my dagster.yaml and jobs details where i'm setting the tag. But when I run the Dagster all the jobs starts synchronously by sensor instead of only two.
Copy code
run_coordinator:
  module: dagster.core.run_coordinator
  class: QueuedRunCoordinator
  config:
    max_concurrent_runs: 10
    tag_concurrency_limits:
      - key: "Reports"
        limit: 2
job details
Copy code
@job(
  resource_defs={"report_details": report_details.get_details},
  tags={"type": "Reports"},
)
j

johann

11/12/2021, 1:34 PM
It looks like the tag on your job doesn't match the limit. Try:
Copy code
run_coordinator:
  module: dagster.core.run_coordinator
  class: QueuedRunCoordinator
  config:
    max_concurrent_runs: 10
    tag_concurrency_limits:
      - key: "type"
        value: "Reports"
        limit: 2
q

Qumber Ali

11/12/2021, 1:35 PM
Tried this too
it's not working for me.
j

johann

11/12/2021, 1:36 PM
How are you deploying dagster? Does your daemon have the same dagster.yaml configured?
Some logs from the daemon process could be revealing
q

Qumber Ali

11/12/2021, 1:37 PM
I'm running these two lines to run the dagster.
dagit -f repositories/ss_jobs_repo.py
dagster-daemon run
Solved, thanks.
j

johann

11/12/2021, 1:51 PM
What did it end up being?
q

Qumber Ali

11/12/2021, 1:52 PM
I didn't add any configurations my ~/dagster_home?dagster.yaml
I'm adding in repo/dagster.yaml.
2 Views