Hi Team, Regarding dynamic partitioning, is there a way to define fixed size partitions (e.g. 1000 records) for a large set of ids? My use case is more of to use partitioning to divide a large dataset into smaller fixed size chunks to be processed by functions that might be expensive, so that the whole process does not run out memory or take exceptionally long to run

Hi Dion. There isn't a built in way to enforce that each partition only targets a fixed number of records, but you could do that internally within your asset. I.e. you could define dynamic partitions like

1-1000

,

1001-2000

, etc. and then within your asset use those keys to determine which record IDs can be targeted

Hi, do you have any sample code that does this?