https://dagster.io/ logo
#ask-community
Title
# ask-community
t

Timur Iskhakov

07/19/2023, 6:30 AM
I have a question somewhat similar to Stefan's post above. I have a resource that fetches data from an external service. Since the data changes quite rarely, I'd like to cache it within the resource's state. However, is seems that the resource's lifecycle is rather short as it's being created anew with each run. I wonder if there's a way to extend the resource's lifetime and make it shareable across multiple runs?
dagster bot answered by content 1
Long story short - no it is not possible. You could cache the data locally to wherever you're running Dagster and have the resource be a gateway to that cache, but you can't share a single resource instance across multiple steps or runs
The exception to sharing across multiple steps would be if you use the in-process executor
t

Timur Iskhakov

07/19/2023, 5:00 PM
Thank you for your response! Appreciate suggested workarounds. Caching locally was my plan B, so I'll go with that then.
c

claire

07/19/2023, 9:46 PM
+1 to Zach's comments, I added a discussion here that details this: https://github.com/dagster-io/dagster/discussions/15398
👍 1