Dane Linssen05/03/2023, 7:23 AM
: • According to the migration document, migrating to
adds primary key id columns to the
tables: https://github.com/dagster-io/dagster/blob/master/MIGRATION.md#database-migration • We followed this manual: https://docs.dagster.io/deployment/guides/kubernetes/how-to-migrate-your-instance • After a successful job in k8s …
… we still don’t have primary key id columns in the
kubectl get po -n dagster NAME READY STATUS RESTARTS AGE ... dagster-dagit-instance-migrate-lmgz9 0/1 Completed 0 2m24s ...
tables. These are the logs:
It skips steps as “they have already been applied”. This is not currently an issue for us, just wanted to report back as this might be valuable information for you 🙂. Thanks for everything that you’re doing! Cheers!
Updating run storage... Skipping already applied data migration: run_partitions Skipping already applied data migration: run_repo_label_tags Skipping already applied data migration: bulk_action_types Updating event storage... Skipping already applied data migration: asset_key_index_columns Updating schedule storage... Skipping already applied migration: schedule_jobs_selector_id
prha05/03/2023, 8:21 PM
table? And can you confirm which storage you’re using?
Tim van Druenen05/04/2023, 9:08 AM
About which storage we're using, I guess you want to know which database (version) we're running. I've tried the db migration with two different deployments of dagster with two different databases: 1. very basic Postgres 11 (deployed by Zalando Operator) 2. Postgres 14.6 incl. pgBouncer (deployed by Stackgres Operator) All in Kubernetes with active/standby (replication) enabled. I used this query to verify if there are primary keys available:
SELECT * from public.alembic_version; version_num -------------- d9092588866f (1 row)
The output before and after the migration was the same:
select tab.table_schema, tab.table_name from information_schema.tables tab left join information_schema.table_constraints tco on tab.table_schema = tco.table_schema and tab.table_name = tco.table_name and tco.constraint_type = 'PRIMARY KEY' where tab.table_type = 'BASE TABLE' and tab.table_schema not in ('pg_catalog', 'information_schema') and tco.constraint_name is null order by table_schema, table_name;
You can see the full output of the migration pod attached. Thanks for your help!
table_schema | table_name --------------+------------------- public | daemon_heartbeats public | instance_info public | kvs (3 rows)
prha05/04/2023, 1:23 PM
Tim van Druenen05/05/2023, 7:31 AM
prha05/05/2023, 1:02 PM