02/09/2022, 9:06 AM
With the upcoming 0.14 edition of dagster will partitioned (software-defined) assets be part of the API?
02/09/2022, 5:03 PM
02/09/2022, 5:04 PM
yup! in fact, partitioned assets are available in preview in our most recent release
02/09/2022, 5:06 PM
I know. But so far I did not figure out how to use partitions in software defined assets. Would you have an example for me?
02/09/2022, 5:09 PM
it's not the most concise example, but the hacker news assets demo contains partitioned assets. here's a definition for one:
and here's a job that includes some:
I'd be happy to walk through it on a zoom if it would be helpful
02/09/2022, 5:17 PM
Thanks - I think the two links are great examples to what I need.
However, how is the mapping of partitions performed from one asset to the next one (downstream in the lineages). Is it a must, that they are all partitioned in the same way?
02/09/2022, 5:29 PM
currently, all the assets within the same job must have the same partitioning (or no partitioning). we have a more flexible notion of a `PartitionMapping`:
, an important subclass of which is `TimeWindowPartitionMapping`:
however partition mappings currently only work across jobs
02/09/2022, 6:27 PM
Do you have any plans to support a mix of non-partitioned and partitioned assets?
I.e. a daily/hourly job with enrichment of some non-partitioned metadata
02/09/2022, 6:28 PM
with our current apis, you can combine partitioned and non-partitioned assets in a single job