We've been using dagster for a few months, and have accumulated enough jobs that we need to do some reorg-ing of one massive repository to multiple smaller repositories. If I change which repository a given dagster job is a part of, does that retain the run/partition history? I did a rename locally and looked like it carried history over, but just wanted to get confirmation before pursuing this...especially since I don't have partition history locally

Hi Adam. What version are you running? As of

1.0

this is no longer true. We used to show runs of identically named jobs, independent of repository, but as of

1.0

we only show runs that were created by jobs for that specific repository. Ideally we’d have some run migration CLI tools, but I can walk you through how to do those run migrations by running a data migration against your DB if you’re interested.

We're on 0.15.8, but I plan to check out 1.0 later this week (not sure the private module changes will work with our custom executor and partition selector). I figured a migration might be necessary - totally willing to write/run one!

All runs (that are up to date on run migrations) should contain a hidden run tag in the

run_tags

table with key

.dagster/repository

and value of the form

<repository_name>@<location_name>

… We use this value to filter by the job’s current repository. If you move the job to a differently named repository, it won’t be able to find the historic runs unless you run an update to the new repository name. If you’re splitting the mega-repo into smaller repos, you may have to join the

run_tags

against the

runs

table to filter by job name to get the appropriate new repository name.

Created issue at: https://github.com/dagster-io/dagster/issues/9317

Thanks @prha! I'll take a look at this soon