Does anyone else experience the problem with “live” event logs? The problem is I stopped receiving the live updates from pipelines, and only see changes after page refresh. Websocket connection is normal, but I don’t see any messages flowing. I use Dagit 1.0.15 behind Nginx, will try to downgrade and check again.

I’ve seen this behave intermittently like you described, across numerous dagster versions

which database are you using? there is a “disable websockets” toggle in the settings page that can be used to force polling

which database are you using?

postgresql in docker

you see any error logs for the dagit process?

I rebuilt the image with dagster==1.0.15 and it works now 🤔

there was a recent change https://github.com/dagster-io/dagster/commit/e8d8525bfd01d9f283148a0da3bd3b25271a16b2 so one possibility is that certain

sqlalchemy

versions have problems? unclear it could also be this sporadic issue that Adam mentions which im not sure what to make of . One speculation is that its related to our old implementation stuck on an ancient version of

rx

which i am hoping to update to async generator backed https://github.com/dagster-io/dagster/pull/9334?no-redirect=1

@alex I was hoping that change would help with the "Unexpected GraphQL Error"s (postgres), but no luck. still getting sqlalchemy

(psycopg2.OperationalError) SSL SYSCALL error: EOF detected

pretty much daily

hmm, do you have

keepalive

related kwargs set in addition to enabling ssl?

yup, we added that months ago, but no luck. Seems to only happen when dagit has been idle for a while (i.e. first use of the day). Pretty sure the database has long closed connections as idle that sqlalchemy is attempting to use.

looks like we already have

pool_recycle

on for mysql dagit connection - could add it to pg storages

sqlalchemy docs explicitly mention mysql and not postgres for that param, but also sounds like it would do what we want? https://docs.sqlalchemy.org/en/14/core/pooling.html#setting-pool-recycle

you down to submit one?

I can definitely do that

one argument is that since the mysql variant is always set it should be fine in PG as well- changing out connections once an hour seems pretty reasonable making a

dagit

cli flag to set it is probably ideal but a lot more plumbing. If we did that i think we would still have the default be the same

3600

s in both

@alex wasn't too much plumbing! https://github.com/dagster-io/dagster/pull/10416