How well does dagster work concurrently? On github issue 4041 it shows async as an open issue. So I'm guessing still working on concurrent processing and already done with parallelism. Unless this has been moved to dynamic output on which I'm a little fuzzy. Let me know what I'm missing.

We have support for multiprocess execution, where we can execute ops concurrently (this is the default executor). Issue #4041 is tracking the async executor which executes the ops in a single process using threads. This avoids the process spin-up cost for each op. It’s still very much on our roadmap but we have some conflicts with the current implementation of the compute log manager, which we’ll need to rewrite. I know some users have used the

NoOpComputeLogManager

(to basically disable the compute log manager) and written their own async executor successfully to achieve the same effect.

That's a detailed and concise answer. I clearly have a lot to learn from you.