Hey there! Can anyone help with this error I get w...
# ask-community
y
Hey there! Can anyone help with this error I get when executing a pipeline?
Copy code
sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) Failed processing pyformat-parameters; Python 'pendulum' cannot be converted to a MySQL type
[SQL: UPDATE runs SET `status`=%(status)s, run_body=%(run_body)s, update_timestamp=%(update_timestamp)s, start_time=%(start_time)s WHERE runs.run_id = %(run_id_1)s]
[parameters: {'status': 'STARTED', 'run_body': '{"__class__": "PipelineRun", "asset_selection": {"__frozenset__": []}, "execution_plan_snapshot_id": "00fa4f5ffdc965990dadefa6471507a4a8a2396b", "ext ... (3958 characters truncated) ... rucor_FSOS_By_SceneType_By_SKU_1608", "write_vanilla_calculations", "combine_kpi_results", "generate_result_hierarchy", "write_results"], "tags": {}}', 'update_timestamp': <Pendulum [2023-05-22T12:23:58.215929+00:00]>, 'start_time': 1684758238.215929, 'run_id_1': '12ef4820-d9eb-4975-9b17-ba09b8ea247b'}]
It's happening in the execute_in_process and execute_job methods when it's trying to update the runs table in the database. I see here :
dagster/_core/storage/runs/sql_run_storage.py:206
that the update_timestamp field is set as a Pendulum object. but from some reason the MySQL cannot handle that. I recently installed pendulum==1.4.4 in my environment to solve a different issue, perhaps it's related? Thanks!
o
Hi @Yoav Gilor! My suspicion is that this has something to do with your mysql-connector version, as this is what is in charge of doing this conversion. Pinning the pendulum version was likely the catalyst for whatever went wrong, but dagster is compatible with (and tested against) pendulum versions < 2 and > 2, so in theory that should still work.
y
Hey @owen, Thanks for the reply! Which module are you referring to? mysql-connector-python or pymysql? I see I have both of them
o
mysql-connector-python
y
Hey @owen, Can you recommend a version which runs well with Dagster 1.3.2 ? I've tried some and either got the same error or got other version mismatch errors
o
I have
mysql-connector-python==8.0.31
-- if you do a fresh install of dagster in an empty virtualenv, do you still get this issue?
153 Views