https://dagster.io/ logo
Title
m

Marco Jacopo Ferrarotti

11/09/2022, 11:15 AM
Hi, I'm testing dagster in a mixed cloud/hpc environment and I would like to write a custom run launcher to launch jobs on the hpc partition which is governed by a workload manager (PBS).. Is there any kind of documentation and/or blog post explaining how to implement a custom run launcher?
d

daniel

11/09/2022, 8:22 PM
Hi Marco - we don't currently have docs for this unfortunately. My recommendation to start would be to look at the DockerRunLauncher as an example: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-docker/dagster_docker/docker_run_launcher.py#L141 The main difference would be that it would need to run that same command in a container in some other environment If more detailed docs would be helpful here would you find filing a github issue to help prioritize?
Also happy to talk through questions that come up here from looking through that run launcher as a nexample