Two questions about partitioning:
1. so a static partition needs to be defined with a fixed list of partitions, but can I later add more or will that crash/compromise the asset?
2. can somebody offer an example of how an unpartitioned downstream asset can be made dependent on a single partition of another asset? (i.e. if the partitioned asset corresponds to a particular version of something, or a machine learning model, etc)
02/17/2023, 5:42 PM
hi! for (1), you can add more partitions to the definition later without any issues. for (2) the only supported partition mappings can be found on this page, so unfortunately there's nothing there that does exactly what you want. However, I think if you set up your unpartitioned asset as an asset with a single partition, you could use either the IdentityPartitionMapping, or the StaticPartitionMapping to do what you want there
02/19/2023, 7:42 AM
nice thanks, I can see how a single partition might make the most sense to ensure a pretty generic+extensible design, even if it might also look a bit silly in the code, glad there's a solution 🙂
Might be a good addition to the docs.