I am looking into creating a step launcher for dat...
# ask-community
p
I am looking into creating a step launcher for dataproc. One question I have is how do you specify that an op / job should use the step launcher that you provide it?
z
you just configure it as a resource on the job, and any ops that require the resource key tied to the step launcher will use it automatically. like this:
Copy code
@op(required_resource_keys={"remote_launcher"}
def remote_execute():
    pass

@op
def local_execute():
    pass

@job(resource_defs={"remote_launcher": step_launcher_instance"})
def job():
   remote_execute()
   local_execute()
Dagster inspects the resources for each op and knows to use the step launcher if an op has a step launcher configured for a resource
not sure how you do it with the new pythonic config, but I'd guess it's similar - just pass in a step launcher resource and dagster will figure it out
p
So dagster is able to figure out that the resource is a step launcher and automatically uses it as such?
s
exactly
❤️ 1