Hi all, hope you're having a nice Friday I'm havin...
# ask-community
s
Hi all, hope you're having a nice Friday I'm having a bit of an issue using a Resource that gets instantiated twice in a job, the resource signs into a service and returns an object to interface with said service, but calling that sign-in a second time causes the first instance (in a running op, since both ops that use the resource run at the same time) to fail. The docs for resources say
Depending on your executor, resources may be instantiated and cleaned up more than once in a job execution.
, is there anywhere I can read more on this to better understand how and when resources are instantiated? This is a simple @ job with a couple @ op and the single @ resource in the same file
o
hi @Salvador Ribolzi! by default, jobs will run with the
multiprocess_executor
, which will run each op in its own separate process. This means that ops will each have to instantiate their own separate copies of a resource when their process is initialized (as the resource object can't be shared across the process boundary). For simple local use cases, you can swap the execution mode such that all ops run in the same process: https://docs.dagster.io/concepts/ops-jobs-graphs/job-execution#default-job-executor
z
Is the object that interfaces with the external service pickeable? you might be able to set it up / do the sign-in in an upstream op, then pass that object as an Output to any downstream ops that rely on it.
1