Let’s say I have a single job in which half the ops can run on the main k8s job and over half the ops that need to be spun out as their own k8s job. Whats the best way to do this? Is it to use the default k8s executor and create a manual

k8s_job_op

/`execute_k8s_job` for every desired isolated container? Is that a way to create a container per op by default? And is there a convenient way to spin up containers that by default use the same image as the main dagster job?

using https://docs.dagster.io/_apidocs/libraries/dagster-k8s#dagster_k8s.k8s_job_executor in combination with the https://docs.dagster.io/_apidocs/libraries/dagster-k8s#dagster_k8s.K8sRunLauncher might be what you're looking for the RunLauncher will create a k8s job for each dagster JobRun, the. within that job the k8s_job_executor will create more k8s jobs for every step(op)

i think what i was hoping for, was something along the lines of…

@op
def return_five():
    return 5

@op(executor=k8s_executor)
def add_one(arg):
    return arg + 1

@job
def do_stuff():
    add_one(return_five())

defs = Definitions(
    jobs=[do_stuff], executor=multiprocess_executor
)

you understand it correctly, currently the lines between the different abstractions for execution are a bit blury