is there a way to do “depth-first” instead of “bre...
# ask-community
e
is there a way to do “depth-first” instead of “breadth-first” with
DynamicOut
? I have a case with ~50k items that are dynamically loaded from a database and need to be processed by ops that I map, but right now Dagster enumerates all 50k via
DynamicOut
before processing the first one. this leads to super expensive startup times for the job, when I really want it to enumerate a few, process them, then enumerate more
dagster bot responded by community 1
q
❤️ 2
d
that discussion will help with the order in which the ops are processed once the dynamicouts are returend - but I don't believe it will help with the 'enumerate a few, process them, then enumerate more' part
e
quick update: playing with priority ended up solving it if A spawns B spawns C, set priority of A = 1, B = 2, C = 3. B and C will run before the next A!
🎉 2