https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jorn

03/10/2023, 12:16 PM
Hello after redeploy with version 1.1.17 and even after update to 1.2.0 I am having issues with the daemon. Maybe anyone have an idea how can I fix that? From the logs I get
Copy code
2023-03-10 12:11:23 +0000 - dagster.daemon.BackfillDaemon - ERROR - Failed to add heartbeat: 
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column daemon_heartbeats.id does not exist
LINE 1: ...": [], "timestamp": 1678450283.288276}' RETURNING daemon_hea...
                                                             ^

[SQL: INSERT INTO daemon_heartbeats (daemon_type, daemon_id, timestamp, body) VALUES (%(daemon_type)s, %(daemon_id)s, %(timestamp)s, %(body)s) ON CONFLICT (daemon_type) DO UPDATE SET daemon_id = %(param_1)s, timestamp = %(param_2)s, body = %(param_3)s RETURNING daemon_heartbeats.id]
[parameters: {'daemon_type': 'BACKFILL', 'daemon_id': 'f24f7a9b-50de-4b88-ba8e-3c5396f83159', 'timestamp': datetime.datetime(2023, 3, 10, 12, 11, 23, 
288276, tzinfo=datetime.timezone.utc), 'body': '{"__class__": "DaemonHeartbeat", "daemon_id": "f24f7a9b-50de-4b88-ba8e-3c5396f83159", "daemon_type": "BACKFILL", "errors": [], "timestamp": 1678450283.288276}', 'param_1': 'f24f7a9b-50de-4b88-ba8e-3c5396f83159', 'param_2': datetime.datetime(2023, 3, 10, 12, 11, 23, 288276, tzinfo=datetime.timezone.utc), 'param_3': '{"__class__": "DaemonHeartbeat", "daemon_id": "f24f7a9b-50de-4b88-ba8e-3c5396f83159", "daemon_type": "BACKFILL", "errors": [], "timestamp": 1678450283.288276}'}]
(Background on this error at: <https://sqlalche.me/e/14/f405>)

Stack Trace:
  File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/daemon.py", line 109, in run_daemon_loop
    error_interval_seconds,
  File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/daemon.py", line 174, in _check_add_heartbeat
    errors=[error for (error, timestamp) in self._errors],
  File "/usr/local/lib/python3.7/site-packages/dagster/_core/instance/__init__.py", line 2339, in add_daemon_heartbeat
    self._run_storage.add_daemon_heartbeat(daemon_heartbeat)
  File "/usr/local/lib/python3.7/site-packages/dagster_postgres/run_storage/run_storage.py", line 189, in add_daemon_heartbeat
    "body": serialize_dagster_namedtuple(daemon_heartbeat),
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 335, in _execute_on_connection
    self, multiparams, params, execution_options
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1582, in _execute_clauseelement
    cache_hit=cache_hit,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1944, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2125, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

The above exception was caused by the following exception:
psycopg2.errors.UndefinedColumn: column daemon_heartbeats.id does not exist
LINE 1: ...": [], "timestamp": 1678450283.288276}' RETURNING daemon_hea...
p

prha

03/10/2023, 4:16 PM
Hi Jorn. Thanks for the report! I think this is a bug introduced in
1.2.0
. I’ll try to get a patch out. One thing that could fix this is to run a db schema migration by running
dagster instance migrate
, if you are able to.
This should now be addressed with the just released
1.2.1
j

Jorn

03/13/2023, 7:57 AM
Hello Phil Yes it is resolved now. thank you very much
3 Views