Anybody else using Postgresql to store runs, etc.? The Dagit instance has been hanging the server and the only exception I can see is this:

Exception in thread postgres-event-watch:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/kfdev/dagster/v_dagster/lib/python3.8/site-packages/dagster_postgres/event_log/event_log.py", line 258, in watcher_thread
    for notif in await_pg_notifications(
  File "/home/kfdev/dagster/v_dagster/lib/python3.8/site-packages/dagster_postgres/pynotify.py", line 123, in await_pg_notifications
    conn.poll()
psycopg2.OperationalError: SSL SYSCALL error: Connection timed out

Anyone have seen this?

Are you using docker/docker-compose for the deployment at all? I had something similar and updating Docker fixed the issue

I'm running dagster inside an EC2 server. Just install from python3 pip inside a virtual env. @Cameron Gallivan

Yea I’m running on EC2. I’m using docker-compose (based off the deployment example) to run separate containers for dagit, daemon, and the pipelines/grpc. I used to have another container for postgres but it was getting bottlenecked and I moved to using a small Postgres RDS instance for run/schedule/event_log storage

@Cameron Gallivan thank you for the info! What size of EC2 are you using? I'm using a t3.medium (4GB memory) and I'm wondering if it's too small to run dagster and dagit.

I'm running the dragster deployments on a c6i.xlarge (4 vCPU and 8 GB) since all of my jobs are CPU limited. I cap it at 20 jobs running at a time. But t3s work on bursting cpu credits, COULD they getting depleted and the CPU is getting throttled when the error happens?