19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hello, I’m trying to implement a web app that subm...
# announcementsc
Cameron Gallivan
01/26/2021, 5:22 PMHello, I’m trying to implement a web app that submits a pipeline to be executed to the QueuedRunCoordinator that runs in the background without hanging until the pipeline is complete. Is there a way to implement this with just the Python API or is the best way to just wrap a call to dagster.cli.pipeline.execute_launch_command? The pipelines themselves will probably be fairly lightweight but some solids will be submitting and waiting for a longer running job to finish on AWS Batch.
d
daniel
01/27/2021, 2:11 AMHI Cameron - annoyingly execute_launch_command doesn't go through the QueuedRunCoordinator currently. Let me see if we can get something in for the release on Thursday that fixes that.
c
Cameron Gallivan
01/27/2021, 5:03 AMOk thanks, that'd be really helpful. So theres no way to implement the queue and concurrency limits based on tags through the Python API? I'm still working through the docs and just to make sure I'm not missing something
d
daniel
01/27/2021, 5:08 AMwe don't have a great Python API entry point for launching runs (as opposed to executing them) right now, yeah - it's something we'd like to do but haven't gotten around to yet. execute_launch_command is a bit of a workaround for that in the interim. It should be a quick fix to make it so that execute_launch_command goes through the path that uses the queue/concurrency limits when they're configured on your instance.
daniel
01/27/2021, 5:10 AM(the distinction between launching and running is that executing executes your pipeline right there in the process, no matter how your instance is configured - that's what the execute_pipeline function does in the Python API. Launching is supposed to go through the same path as dagit, where we use the run launcher/coordinator configured on your instance - it's just a bit of an oversight that it isn't using the queue path right now)
daniel
01/27/2021, 5:16 AMoh and since it's not very well documented - the second argument to execute_launch_command is the argument map for the 'dagster pipeline launch' command. (but with any dashes turned into underscores, so pipeline_name not pipeline-name). Sorry for the rough edges here!
4 Views