Hi friends, I have an asset that runs 300 loops. All these loops can exist as independent assets since they do not depend on each other. Is there a way to gain some efficiencies using dynamic assets or ops? If yes, how can I get this done and lastly will this be the efficient way to go about this?

If you write an asset that is a single iteration of the loop, you could then generate that asset in a loop 300 times (giving a different name to each one with the

name

parameter of

@asset

and likely some different values in the asset function itself). That would give you 300 assets that you could add to your Definitions object. If you were to materialize all of those assets at once, you’d be able to take advantage of parallel execution

Any example I can follow?

Thanks @jamie

no examples of that specifically, but when you write the asset you could specify the

ins

parameter programmatically https://docs.dagster.io/concepts/assets/software-defined-assets#defining-explicit-dependencies you might end up needing to have three loops, one for A, one for B, one for C if specifying the

ins

for each is significantly different

Thank you so much! I'll give it a try