Quickly checking, is there anything that supports ...
# ask-community
a
Quickly checking, is there anything that supports retaining a history of asset materializations? Our use case: we have an external source we are querying that we want to keep entire snapshots of over time.
🤖 1
dagster bot responded by community 1
If we implement this ourselves, are there recommendations? Dynamic
AssetKey
with the date?
a
A partitioned asset may be what you’re looking for. https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions If the snapshots occur on a regular time interval, those could be your partitions
a
We are already using partitioned assets but not sure it’s appropriate in this case. The upstream resource changes day to day and we want to snapshot moments in time. If we set it up as partitions, it would break if we ever reran a partition by accident (since it will be replaced with the current days info)
a
you could add a check at the start of your asset/op that the partition key is valid to prevent that
a
So basically if partition key != today then NOOP
s
I don't have a better answer than @Adam Bloom, but I filed an issue to track supporting this in a more first-class way: https://github.com/dagster-io/dagster/issues/12253
a
Thank you 👍🏼