As code develops, some assets might get updates. T...
# dagster-feedback
As code develops, some assets might get updates. The new code version functionality supports this and allows for detecting necessary rematerializations. But in many settings we need to keep the persisted copies of the older assets around for reproducibility and it'd be nice to also have a record of their lineage even if the updated asset has seen changes in that aspect. How does/could Dagster and the SDA approach assist with bookkeeping of versioned and historical assets? I guess the expected way of managing it is to come up with a new key for each new version, but this gets a bit untenable in the long run. Incidentally, cutting the period character out of your accepted character set really limits the ability to apply semantic versioning as e.g. part of a key prefix without obscuring string sanitation (
), so this might just be a long-winded argument for adding that 🙂 but it could also be convenient to be able to "retire" or "archive" assets when their definition can no longer be maintained but the lineage remains relevant, or just to be able to add lineage to something like a source asset
another use case - when a version represents an output data product, sometimes we'll need to run multiple versions in parallel to allow for transition time. and then we may or may not need to backfill older partitions (the change may be adding a column that is only present in new data, as an example). I need to do some thinking on how to best solve this within our current model - looked into code_version on assets, but like Rasmus, I don't think it works for this case