I'm having two issues in Dagit (0.13.9) with a large DAG (63 ops): • The page takes several minutes to load in waterfall view or timed view. (The job overview loads just fine.) If I prune 10% of the graph off, it works fine, and as I add more back in it gets way worse quickly, I'm wondering if there's an n^2 in a visual layout loop or something? It shows "Fetching logs" with a spinner while it's loading. I can load other Dagit pages just fine while the waterfall view is blank w/ a loading spinner, and Firefox or Chromium on Linux show me a 'the page is slowing down the browser' message. Same loading a running or completed job. • If I have scrolled down and to the right in the waterfall view (after it loads), when the graph status updates as ops complete, the view scrolls rapidly back up and to the left. Known issues? What do you need for a good bug report? My graph definition has a bunch of hooks into application logic that would be kind of a pain to strip out to send over an example, is there an intermediate DAG representation I could send?

Hi Mark - there have actually been a fair amount of improvements to the performance of the run page since 0.13.9 - if upgrading to the latest version of 0.13 is an option I'd be curious if that makes it better, and there shouldn't be any breaking changes from the upgrade.

Thanks Daniel, I upgraded to 0.14.3. (Still early development, no reason not to stick with the latest except that we have a practice of pinning versions.) Definitely much better! The scrolling issue is fixed. It still takes about a minute to load initially (even on the flat view), but not nearly as long.

That's great that there was some improvement- do you possibly do a lot of context.log calls in your ops? We've seen some slowness still for runs that do a lot of logging

It doesn't feel like a lot. (: What's an approximate threshold and a way to evaluate it? I only see about 10 calls to context.log in the pipeline, and none of them are in loops getting called a bunch, so I think it's pretty safe to say no.

Yeah 10 should be no problem unless they're, like, gigabytes of data each. Good to know, thanks

I just added some extra inputs to existing ops, and loading time got much worse again.

Is exporting the run to a file from dagit an option so we can take a look? That would share the logs with us for the run and the structure of the job, but none of the actual code or data

Will do!

If I prune 10% of the graph off, it works fine, and as I add more back in it gets way worse quickly

this sounds like the threshhold for background service worker driven render cc @Ben Gotow

hey folks! thanks for filing this Mark - it looks like this particular DAG shape was causing infinite recursion in our layout algorithm. I tracked it down this morning and put up a PR for review - we’ll try to get this in to the next release! 👋

Hi @Ben Gotow, I updated to Dagit 0.14.5 (and other py packages) and unfortunately the Dagit UI still hangs for my DAG in the run view. I updated https://github.com/dagster-io/dagster/issues/6942 with a debug download from my latest run attempt.

ahh that’s super strange - thanks Mark, let me see what happened 😞

Hi @Ben Gotow, just checking if this went in to 0.14.6. Looks like your commit went in Thursday and that's when the release also went out, so probably it'll be available in 0.14.7?