The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

image.png

After the last version, the daemon stopped out of nowhere and this log message appeared on logs

what version of `grpcio`  is running here? There were some problematic versions between `1.48.0 &lt; X &lt; 1.56.0` that were known to be problematic

I sent a recent problem on this channel talking about the same problem on version 1.4.2

I'm doing a rollback because it stopped 6 times just this morning

<https://dagster.slack.com/archives/C01U954MEER/p1690385913224109>

&gt; How can I see it?
`pip list | grep grpcio`

I did the rollback to dagster 1.3.14, but still got this error :disappointed:

E0726 12:28:50.807539991 8 <http://original_thread_pool.cc:232]|original_thread_pool.cc:232]> Waiting for thread pool to idle before forking (1 to 0)

I'm still getting problems with the Daemon

heres the `grpc` issue that this is related to
<https://github.com/grpc/grpc/issues/31885>

It looks like you are on `1.56` which we believe to fix the issue. Are you certain where you ran `pip list`  in the same python environment the daemon is running in?

What exactly is your deployment set-up where you are observing this?

I use requirements.txt to evaluate which version to use of anything and yes, Where I ran pip list was at the same env the daemon is running

on what python version and OS/platform is this running?

you could try to pin `"grpcio&gt;=1.44.0,&lt;1.48.0"`

Python 3.9, Linux on openshift, the image is python3.9-slim-bullseye

I'll try to fix the version on 1.44.0 or 1.48.0, thanks