Hiroyuki Ota
04/28/2023, 4:58 AMchris
04/28/2023, 5:33 PMchris
04/28/2023, 5:44 PMHiroyuki Ota
04/29/2023, 2:51 AMThis is definitely not expected. If I had to guess, the code location naming didn’t get preserved, and dagster thinks all of your stuff is completely new.I'll check that whether the name was changed or not.
Here’s an example of converting from repos to defnitions: https://github.com/dagster-io/dagster/pull/11135 - want to make sure there’s no disparities with your migration story hereThank you very much! I'll check it through.
Hiroyuki Ota
04/29/2023, 5:07 AMmy_repo_name@my-code-location
but now it is my-code-location
.
Questions
• The behavior described above (resetting schedules & sensors etc.) is expected when migrating originally named repositories to definitions?
• If so, is there any workaround?
◦ I found create_repository_using_definitions_args, but is it a suitable for this use case? We only have one repository per definition.
More details:
I confirmed that code location name was unchanged by checking that the workspace.yaml file was not changed.
"load_from":
- "grpc_server":
"host": "hota"
"location_name": "my-code-location"
"port": 3030
In a repository before definition migration fetches runs with following.
"variables": {
"filter": {
"pipelineName": "my_pipeline",
"tags": [
{
"key": ".dagster/repository",
"value": "my_repo_name@my-code-location"
}
]
},
"limit": 26
}
The new code location filters runs with a slightly different condition, which I guess this is why we don't see past runs.
"variables": {
"filter": {
"pipelineName": "my_pipeline",
"tags": [
{
"key": ".dagster/repository",
"value": "__repository__@my-code-location"
}
]
},
"limit": 26
}
chris
05/02/2023, 6:15 PM