Hey all, I'm trying to understand partitions and how they are used and if my use case matches that of partitions. I have multiple hosts to connect a client (who's auth needs to be updated every 30 mins) to and run the same ops on (like

process_host_data

) We then need to have a fan out something like -

get_clients -> get_customers_for_each_client -> get_inventory_for_each_customer -> get_inventory_stats_for_each_inventory.

Originally our thought was to use dynamic returns, but you can only do that for 1 downstream op at the moment. Our next thought was to break this into smaller pieces and save each of the results into a cache that the next job would read from. What would be the proper "Dagster" way of doing something like this?

Hi Aaron--partitions are usually a good fit for situations where the fan out is predefined and not calculated at runtime. I think the ideal way to handle this would be to have multiple layers of dynamic outs, but sadly this isn't supported at the moment. We do have an issue to track this: https://github.com/dagster-io/dagster/issues/4364