https://dagster.io/ logo
#ask-community
Title
# ask-community
e

Erik Goldman

04/30/2023, 9:06 PM
is there a way to do “depth-first” instead of “breadth-first” with
DynamicOut
? I have a case with ~50k items that are dynamically loaded from a database and need to be processed by ops that I map, but right now Dagster enumerates all 50k via
DynamicOut
before processing the first one. this leads to super expensive startup times for the job, when I really want it to enumerate a few, process them, then enumerate more
dagster bot responded by community 1
q

Qwame

05/01/2023, 4:57 AM
❤️ 2
d

daniel

05/01/2023, 8:07 PM
that discussion will help with the order in which the ops are processed once the dynamicouts are returend - but I don't believe it will help with the 'enumerate a few, process them, then enumerate more' part
e

Erik Goldman

05/02/2023, 11:07 AM
quick update: playing with priority ended up solving it if A spawns B spawns C, set priority of A = 1, B = 2, C = 3. B and C will run before the next A!
🎉 2