The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

I have a conceptual question, when I materialize an asset that depends on different configuration values (say the versions of an upstream data set or model) and I then want to compare this same asset for different configurations, does dagster allow me to do this easily or do I need to introduce a partition to be able to do so?

hi! for this sort of use case, you'd probably want to use partitions

Do you have experience with using a representation of the compute environment as a partition key, for example, a Docker image hash or so? It's interesting to me to see what changes between different versions of software dependencies which I could capture that way.