https://dagster.io/ logo
#ask-community
Title
# ask-community
q

Qwame

03/23/2023, 6:34 PM
Hi friends, I have an asset that runs 300 loops. All these loops can exist as independent assets since they do not depend on each other. Is there a way to gain some efficiencies using dynamic assets or ops? If yes, how can I get this done and lastly will this be the efficient way to go about this?
j

jamie

03/23/2023, 6:49 PM
If you write an asset that is a single iteration of the loop, you could then generate that asset in a loop 300 times (giving a different name to each one with the
name
parameter of
@asset
and likely some different values in the asset function itself). That would give you 300 assets that you could add to your Definitions object. If you were to materialize all of those assets at once, you’d be able to take advantage of parallel execution
q

Qwame

03/23/2023, 6:50 PM
Any example I can follow?
q

Qwame

03/23/2023, 6:55 PM
Thanks @jamie
Anywhere in the docs I can follow an example of dynamic dependencies where the second 100 depend on the first and the third depend on the second? E.g. groups A, B and C each having 100 assets and then A->B->C?
j

jamie

03/23/2023, 7:02 PM
no examples of that specifically, but when you write the asset you could specify the
ins
parameter programmatically https://docs.dagster.io/concepts/assets/software-defined-assets#defining-explicit-dependencies you might end up needing to have three loops, one for A, one for B, one for C if specifying the
ins
for each is significantly different
q

Qwame

03/23/2023, 7:06 PM
Thank you so much! I'll give it a try
hi @jamie This does exactly what I want
The problem is that the my first op will need to read output from an asset
How do I do this?