There actually isn t a celery run launcher you re probably l

Question

There actually isn t a celery run launcher you re probably looking for the celery executor More info here about executing on celery here An explanation of run launchers and executors here More info about the instance and configuring run launchers here

schrockn · Answer

oh Noah is doing his own fancy run launcher

schrockn · Answer

in order to hack together solids executing in their own containers

schrockn · Answer

i m looking forward to seeing the wizardry slightly smiling face

sashank · Answer

ah my bad that s exciting

Noah K · Answer

This is more for avoiding overload on the dagit container

Noah K · Answer

Since I don t want launching 100 pipelines to fork off 100 copies of the CLI slightly smiling face

Noah K · Answer

Would quickly hit the container resource limit and fall over

sashank · Answer

ah we will have built in support for this in Dagster in our 0 10 0 release cc < johann>

sashank · Answer

more specifically the queueing part not the solids executing in their own containers part

Noah K · Answer

Local queuing will be great out of the box but cramming it in a rabbit queue means I can autoscale the runner containers too slightly smiling face

Noah K · Answer

Elastic all the things all the things