https://dagster.io/ logo
#random
Title
# random
q

Qumber Ali

11/15/2021, 6:25 AM
Hi all, I'm facing the issue of slowness like I have a job which takes 3 to 4 minutes to get complete when I run a single job it completes in 3 to 4 minutes but when I run around 32 concurrent jobs the time for each job reach to 1 hour or more even though my ec2 server is large and using one 20% of its resources only, i'm not sure why it's happening that by increasing the concurrent jobs speed getting slow.
Anyone?
s

sandy

11/15/2021, 5:08 PM
Hi @Qumber Ali - I can't think of a straightforward way that Dagster would be causing this slowdown. Are you able to tell where the bottleneck is? I.e. what part of the jobs are running slow?
q

Qumber Ali

11/16/2021, 2:54 PM
@johann Can you please have a look at this too?
j

johann

11/16/2021, 2:55 PM
It looks like Daniel responded to your thread in https://dagster.slack.com/archives/C01U954MEER/p1636957546191700
2 Views