https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Mark Fickett

12/13/2022, 6:28 PM
I have an
@op
yielding `DynamicOutput`s that's taking 30m to yield 5600 outputs, taking about 0.5s between each "Yielded output" log message both when it started and when it was finishing. Is this expected? Running on k8s. I wouldn't mind except it doesn't seem like the outputs start running until the op finishes. The op is pretty simple:
Copy code
@dataclass
class Test:
   test_id: str
   # etc

@op(out=DynamicOut(Test))
def yield_each_test(test_list: List[Test]):
    for test in test_list:
        yield DynamicOutput(value=test, mapping_key=test.test_id)
Is there a way to structure the
@op
more efficiently?
j

jamie

12/13/2022, 7:14 PM
cc @alex
thankyou 1
a

alex

12/13/2022, 7:39 PM
you would have to use a profiler to discern the exact cause of slowdown - i would guess its the event writes to the database
m

Mark Fickett

01/03/2023, 9:05 PM
I filed this ticket to track the issue: https://github.com/dagster-io/dagster/issues/11475 . (Jamie and Alex, I think what you're saying is that the slowdown is likely on the Dagster framework side, it doesn't look like my op is doing something obviously slow.)