Hi, dagster team! We've migrated from repository t...
# ask-community
h
Hi, dagster team! We've migrated from repository to definitions. After that 1. All of our schedules and sensors were disabled. 2. Cursors of sensors were lost and all jobs reran. 3. History is not shown in the jobs page. Is it expected? Is there any way to work around the issue?
c
This is definitely not expected. If I had to guess, the code location naming didn’t get preserved, and dagster thinks all of your stuff is completely new.
Here’s an example of converting from repos to defnitions: https://github.com/dagster-io/dagster/pull/11135 - want to make sure there’s no disparities with your migration story here
h
Thank you!
This is definitely not expected. If I had to guess, the code location naming didn’t get preserved, and dagster thinks all of your stuff is completely new.
I'll check that whether the name was changed or not.
Here’s an example of converting from repos to defnitions: https://github.com/dagster-io/dagster/pull/11135 - want to make sure there’s no disparities with your migration story here
Thank you very much! I'll check it through.
Hmm, I think code location name was not changed. We named the original repository but maybe in the definitions we cannot give a name to a definition? In the UI, the code location was shown as
my_repo_name@my-code-location
but now it is
my-code-location
. Questions • The behavior described above (resetting schedules & sensors etc.) is expected when migrating originally named repositories to definitions? • If so, is there any workaround? ◦ I found create_repository_using_definitions_args, but is it a suitable for this use case? We only have one repository per definition. More details: I confirmed that code location name was unchanged by checking that the workspace.yaml file was not changed.
Copy code
"load_from":
- "grpc_server":
    "host": "hota"
    "location_name": "my-code-location"
    "port": 3030
In a repository before definition migration fetches runs with following.
Copy code
"variables": {
  "filter": {
    "pipelineName": "my_pipeline",
    "tags": [
      {
        "key": ".dagster/repository",
        "value": "my_repo_name@my-code-location"
      }
    ]
  },
  "limit": 26
}
The new code location filters runs with a slightly different condition, which I guess this is why we don't see past runs.
Copy code
"variables": {
  "filter": {
    "pipelineName": "my_pipeline",
    "tags": [
      {
        "key": ".dagster/repository",
        "value": "__repository__@my-code-location"
      }
    ]
  },
  "limit": 26
}
c
Hey sorry for the late reply here; we have a solution for you! Made a discussion detailing how to move the history over: https://github.com/dagster-io/dagster/discussions/14049
🌈 1