hi there, I’m interested in merging a dagster postgres database into another pre-existing dagster postgres database. Are there any docs/tips/checklists that can help with this? My current plan is: 1. try ensure they are on the same alembic version 2. pg_dump old db 3. insert the contents into the target database

hmm, i think theres a good chance that you are the first person to try to do this. • the only thing i think that has a risk of colliding is if you used the same asset keys in both - “most recent asset event” data may get weird if you insert/overwrite the asset table • you will have to deal with the auto increment id columns - if the date ranges overlap between the two dbs this may be troublesome. We default to sorting by this id column in various places - so you may end up with some weird product experiences if you have to just add all the inbound records after the existing records depending on how their date ranges interact

thanks, I sense this may not be a fruitful endeavor

yea 😕 if you provide more context on the overall situation i can try to advise

we created a fork dagster environmment to re-architect some pipelines and now want to kill off the fork and return to a single instance. Long-term it helps having access to assets and logs, so we either (a) merge into a single db, or (b) maintain our dead fork potentially forever

how many runs are on the fork

in the region of 1000

we do have

dagster debug export

/

dagster debug import

which wasn’t really designed for this but worth checking out

thanks this looks like the right sort of solution. Any known limitations? (docs are minimal here)

dagster debug --help

is marginally better, not sure whats up on the docs page truncation

I see. Ordering is probs a nice to have. The essential stuff would be the asset catalog (and correct partition dates associated with them)

I believe the partition stuff should all work

thanks this is looking promising

not that i can think of - let me know if anything comes up

just confirming that we migrated 500 runs in order to merge two dagster instances. I reckon this is a use case we should develop for enterprise users. Thanks for the help!