19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
I'd like to ask what's the best practice for large...
# ask-communityw
William
08/09/2022, 6:56 AMI'd like to ask what's the best practice for large DAGs?
A typical use case is to fan out thousands of independent tasks and run them in parallel. In my special case, I have a parameter sweep to run 10k ops using 50 workers (could be on k8s pods or just celery worker) in parallel. However I encounter a few problems:
1. Dagit UI becomes super slow even if I switch off log button (because DAG graph itself is large) -> not a big deal though as I have prometheus+Grafana setup for monitoring.
2. See below screenshot, the job is running well but job status changed to
FailureWilliam
08/09/2022, 9:37 AMhttps://github.com/dagster-io/dagster/issues/7821 might be related
c
claire
08/09/2022, 6:07 PMHi William, thanks for reporting this issue. Which log button are you referring to by "switching off the log button"?
claire
08/09/2022, 6:20 PMYep, the issue you linked to is the cause of the UI being slow. When the run page is loaded in Dagit, all logs are loaded in memory
claire
08/09/2022, 6:46 PMFor the job status changing to failure, would you mind sending over the debug logs for the job? You can access this at the top run page:
w
William
08/10/2022, 2:48 PMWilliam
08/10/2022, 2:48 PMWilliam
09/23/2022, 2:41 AMBump on this
4 Views