The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Good morning. I have a question after scouring the documentation: is there a better way to have a cleanup operation run at the end of a `@job` other than defining an `In(Nothing)` input for the cleanup `@op` and passing it the output of the previous `@op` ?

For context: at the start of my job, I'm creating a temporary directory with `mkdtemp`, and at the end of the job, I want to delete it.

I can't make it a resource because I need the directory to live only for the lifetime of the job.

If there's a better pattern, I'm all ears!

that pattern would definitely work. one thing to consider is that if you’re running your code on something like a kubernetes cluster you may need to do some extra work to ensure that the cleanup op is run on the same vm/pod/etc as the op that creates the directory

ack. we're using dagster-cloud (or dagster serverless) for our deployment