Hi, where can I define the concurrency limits on queues of celeryK8sRunLauncher?

workerQueues

have a

replicaCount

that you can use for this.

So that would limit the number of solids running concurrently for that queue? If replica count is not provided for some queue would it run solids depending on the resources?

So that would limit the number of solids running concurrently for that queue?

Yes

If replica count is not provided for some queue would it run solids depending on the resources?

replica count is required to instantiate a queue, so you cannot omit it

So how would it be possible to run multiple solids based on resources of the k8s system while limiting specific solid with a specific tag (I'm here considering celery queue tags). ?

Ok I'll see into it. There is one more thing, the transition from one solid to another or we can say pod initialization usually takes a lot of time. Like on an average about half a minute. So can we say its a drawback of using celery based k8s system or are there ways to improve them?

sure - the celery workers create kubernetes jobs to run each step, so this is why pods are initialized

Ok thanks for the help.