https://dagster.io/ logo
#ask-community
Title
# ask-community
o

Oliver

06/20/2022, 6:25 AM
Hey, anyone used ray to accelerate dagster pipelines? I'm thinking that i can use ray in my asset definitions and then use the plasma store as an iomanager. I'm not sure if this will work though since I think that after materialisation the process will finish and the ref counter will get decremented and this will result in garbage collection of the object. I am not super confident in my Ray or Dagster knowledge here so this may just work. Otherwise maybe a ray Aactor could be initialised to maintain the ref I guess? Any thoughts about this?
🤖 1
3 Views