how well do partitioned jobs and the associated UI scale to jobs with hundreds of thousands or even millions of partitions? we conceptually have a job that operates on a large set of static partitions and it'd be really cool to be able to use some of the dagster partitioned jobs features to manage and track the partition statuses, but I just wanted to see if anyone had any experience with using it at that kind of scale (maybe that's not even very big?) - I can imagine the UI elements getting a bit unwieldy.

We actually just shipped a new partitions and backfills UI in 0.15 that should help with this. It introduces a new status bar to see status across the entire set of partitions at once. Check it out and let us know what you think. We'd appreciate any feedback on it. 🙏

jeepers that looks nice! I'll have to give it a shot

our current design goal is to support assets with - say - a partition for every hour for a few years, which comes out to 10k or so. hundreds of thousands or millions might be pushing it, though if it ends up working for you, then wonderful