19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
There actually isn’t a celery run launcher – you’r...
# announcementss
sashank
10/27/2020, 10:38 PMThere actually isn’t a celery run launcher – you’re probably looking for the celery executor. More info here about executing on celery here: https://docs.dagster.io/deploying/celery. An explanation of run launchers and executors here: https://docs.dagster.io/overview/run-launchers-executors/run-launcher. More info about the instance and configuring run launchers here: https://docs.dagster.io/overview/instances/dagster-instance#instance-configuration-yaml
s
schrockn
10/27/2020, 10:38 PMoh Noah is doing his own fancy run launcher
schrockn
10/27/2020, 10:38 PMin order to hack together solids executing in their own containers
schrockn
10/27/2020, 10:38 PMi’m looking forward to seeing the wizardry 🙂
s
sashank
10/27/2020, 10:39 PMah my bad, that’s exciting!
n
Noah K
10/27/2020, 10:39 PMThis is more for avoiding overload on the dagit container
Noah K
10/27/2020, 10:39 PMSince I don't want launching 100 pipelines to fork off 100 copies of the CLI 🙂
Noah K
10/27/2020, 10:40 PMWould quickly hit the container resource limit and fall over
s
sashank
10/27/2020, 10:47 PMah we will have built in support for this in Dagster in our 0.10.0 release cc @johann
sashank
10/27/2020, 10:48 PMmore specifically the queueing part, not the solids executing in their own containers part
n
Noah K
10/27/2020, 10:49 PMLocal queuing will be great out of the box, but cramming it in a rabbit queue means I can autoscale the runner containers too 🙂
Noah K
10/27/2020, 10:49 PMElastic all the things all the things
5 Views