I have been poking around with Dagster for about a week to replicate a co-worker's spreadsheet analysis, executed on a frequency that sends notifications based on undesired conditions to MS Teams. I was able to get dagster-daemon and dagit running locally, but I noticed that the pipeline runs significantly slower on dagit (35-40 seconds) compared to when I run it from the command prompt (about 5 seconds). If I watch the execution in the timeline view, the actual solids take very little time, but there is a 750-1000 ms delay between solids. Is there a reason for this, or maybe a setting or approach I could take to speed it up?

hmm interesting are you changing the executor (via the

execution

section of config) between these two approaches? is the

DagsterInstance

set up different between the two invocations? If you haven’t set up

DAGSTER_HOME

the command line would default to in-memory metadata storage where as

dagit

defaults to using the local file system - but this would still be a surprising amount of time added for FS operations