Hi I want to use memoization for my pipeline My ve...
# ask-community
y
Hi I want to use memoization for my pipeline My versioned io manager has dependency on pyspark resource And i see that for each solid in the pipeline new pyspark resource is being created. Is there any way to avoid this? Thanks!
a
i assume this is multiprocess execution? Each process needs its own instance of the resource