https://dagster.io/ logo
#ask-community
Title
# ask-community
s

Samuel Shi

07/17/2023, 8:01 PM
We have a job that exports data every 30 seconds. We want to find if the data has been diffed. To do this I'm thinking of storing the results of the previous run in a new asset and then in the export asset diff the previous results with the new results, then update the new asset to store the current run as the previous run for the next one. However, I'm not sure if this is the best way to do this. Does someone have any thoughts/feedback on this approach?
dagster bot answered by content 1
s

Samuel Shi

07/17/2023, 8:13 PM
Would that be useful if we only want to store one version of previous results, the result immediately before the current one?
r

rex

07/17/2023, 8:15 PM
I believe there is an API to retrieve the existing data version before it is updated — cc @sean
s

Samuel Shi

07/17/2023, 8:17 PM
Would you recommend using an asset to store this previous results data, given that we need to read/write it from an asset?
The other options might be to use a resource to store this data, or to create a new table in our database that can store it