Hello everyone, I was wondering, is there a way to use sequence number as partition instead of dates or static elements? If so is that done by using

DynamicPartitionsDefinition

? This feature is experimental, so I’m wondering if there is another way or if this is the only one. If anyone have a draft of something related to that that would be helpfull too. Thank you 🙂

Hi Jacob, thanks for the question-- I’m a little unclear what you mean here. If you want to use a fixed sequence of numbers, you can use

StaticPartitionsDefinition

. If you don’t know the length of the sequence, then the partitioning scheme is necessarily dynamic, so you’ll need

DynamicPartitionsDefinition

.

It’s a sequence of number but the end is not yet determined. I could use static and create a millions sequential number, but, I’m concern that this may slow down the UI when trying to execute new task or play in the UI. A new partitions (Sequential number) would be created every 10 minutes. I’m better off with DynamicPartitionsDefinition then right?

I’m better off with DynamicPartitionsDefinition then right?

yes

How about creating TimeWindowPartitionsDefinition by passing a cron schedule?