Hey Dagster Community, New to slack - apologies if I am not using any of these features correctly. Is it possible for multiple Ops to share the same instance of a resource? I guess another way of saying that is: is there a way to only instantiate a resource once per job?

Hi @joe! There's no built-in way to do this at the moment. Depending on the type of resource/job you're working with, you might be able to simulate this sort of behavior by doing a sort of "create if not exists" check in your resource initialization function (the thing decorated with

@resource

). It gets tricky because generally jobs are run with a separate process for each step (and these processes might not even be on the same machine), so having the same exact in-memory object for each step might not be possible.

Thank you so much for your response Owen! This is really great to know, I think I've got a couple ideas on how I can recreate this behavior. One really small follow up (my newness to Dagster is showing but I promise I'm an official contributor!), what's the definition of a step? Is that the same as an operation?

the distinction between ops/steps is a subtle one (and I erroneously use them interchangeably a lot of the time), so no worries on not knowing that 😄 . The briefest way to put it is that an op is the definition-time object, and the step is the corresponding execution-time object. In general, you don't really lose much by thinking of them both as the same thing, so feel free to mentally swap out "step" for "op" in my above explanation.