hey- is the InMemoryIOManager broken? It seems the

values

dictionary is not being persisted across different asset classes. Apparently, a separate instance of the class (hence key value store) is being created for every EXECUTION STEP (

STEP_WORKER_STARTED

). Is this expected behavior?

The asset class is not passed through because each step is executed in a new process, super annoying ofcourse when you want to use Ops this way

this is wild

So it only works together with a single executor

yeah thats what i ve reverted to

They basically need to allow to run all ops in a single graph asset in the same process

do you know if multiple jobs that are all defined with a single executor will share the same process?

@Pierre Pasquet that I'm not sure, I think each seperate job has its own process

ok sweet

But that executor is only useful in a local device

i think you are still better off running in a single process than having to deal with I/O operations at each step

Or we need to switch to pushing everything on a single Vm