https://dagster.io/ logo
Powered by
#ask-community
Title
# ask-community
r

Rubén Lopez Lozoya

04/06/2022, 1:44 PM
Hey guys, in Dagster 0.13.1 I've noticed that if you don't set a value for maxConcurrentRuns this defaults to 10. I'd expect no limits to be applied if none is specified, since otherwise I am forced to set an unnecessarily large number so that I can run any number of pipelines at once. I was planning on limiting concurrency using tags and ignoring this param, but I am unable to. Is this something that has been solved in more recent releases? 🙂
j

johann

04/06/2022, 2:29 PM
Hi Rubén, that is silly, I’ll get a fix out
r

Rubén Lopez Lozoya

04/06/2022, 2:44 PM
Cool, are you guys going to fix this for 0.13.x or just bump the current 0.14.x
j

johann

04/06/2022, 2:44 PM
It would only be in the next 0.14.x
r

Rubén Lopez Lozoya

04/06/2022, 2:45 PM
Perfect, also another issue I just noticed in v 0.13.1 is that if you attempt to verride a tag using Dagit UI, it will ask you for the double quotes in the property (tag) name, but if you put the double quote then Dagster appends another pair of quoting marks so the tag is ignored. However, if you do not put the double quotes, it will throw an error asking you for these. Is this solved in latest version?
Copy code
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

  File "/usr/local/lib/python3.7/site-packages/dagster/core/instance/__init__.py", line 1450, in launch_run
    self._run_launcher.launch_run(LaunchRunContext(pipeline_run=run, workspace=workspace))
  File "/usr/local/lib/python3.7/site-packages/dagster_k8s/launcher.py", line 273, in launch_run
    user_defined_k8s_config = get_user_defined_k8s_config(frozentags(run.tags))
  File "/usr/local/lib/python3.7/site-packages/dagster_k8s/job.py", line 148, in get_user_defined_k8s_config
    user_defined_k8s_config_value = json.loads(tags[USER_DEFINED_K8S_CONFIG_KEY])
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
in this example, tag is overriden but pipeline throws an error
And in this other example, you see how adding double quotes implies that tag will not be overriden (in fact, new tag using double quotes will be ignored as Dagster appends another set of double quotes on the run)
j

johann

04/06/2022, 2:56 PM
that error looks to me like the problem is it’s expecting json as the value of the tag
r

Rubén Lopez Lozoya

04/13/2022, 6:15 AM
Hey guys was the issue with maxConcurrentRuns ever solved? I believe I am still experiencing this issue in v0.14.8
j

johann

04/13/2022, 1:53 PM
r

Rubén Lopez Lozoya

04/13/2022, 1:54 PM
❤️
Powered by