Hi team! As I see in docs you may use `-m` flag mu...
# ask-community
m
Hi team! As I see in docs you may use
-m
flag multiple times for
dagster api grpc
(which is used for k8s deployment as well): https://docs.dagster.io/_apidocs/cli#cmdoption-dagster-api-grpc-m But when I try to do that I'm getting the error:
Copy code
dagster._check.CheckError: Invariant failed. Description: Must specify only one code location when executing this command. Multiple {key} options given
Looks like a bug. Is it known?
d
Hey Mykola - this may have been an oversight in the docs that we’ll fix. -m spins up multiple gRPC servers in some commands and it looks like that incorrectly slipped through to the gRPC command as well
m
@daniel, I'm trying to migrate from
@repository
to locations. We have a lot of repositories served under the same grpc server. So what is an intended way for the locations? Should I configure a separate grpc server per location?
this question also can be formulated as "should I create a separate k8s deployment per location" because it looks like there is 1:1 correspondence for deployment:location (because of that
dagster api grpc
limitation)
d
I’m going to tag in @schrockn here to confirm but I think the current answer if you have a lot of repositories in a single code location and don’t want to migrate them to a single Definitions object is to continue using repositories for now
m
Thanks for your answer! Let me add some context to clarify my problems: We mostly use ops/jobs instead of assets for the moment. So we group/isolate the jobs using "repositories". Puling them into one location seems irrational for me at the moment as there is no other mechanism to group jobs inside a single location. I'd like to use location instead of repository as it seems more logical to me especially when migrating our `op`s to `asset`s. But in this case helm chart will look a bit overwhelmed with "deployments" sections. So I'd like to clarify before starting the migration. Some documentation changes are required anyway as there is a discrepancy in the description and behavior.
s
If you want to use the new-style APIs to create named repositories you can use
create_repository_using_definitions_args
in `dagster.code.definitions.definitionsc
m
@daniel, @schrockn, could you please let me know if this is a final decision to have only one "location" per k8s code deployment?
s
So that decision is final. However we will at some point re-introduce mechanisms to organize your definitions within a single code location, but a lightweight and more flexible fashion. In the meantime, the support way to do what you want is to use
create_repository_using_definitions_args
and create named repositories. We will continue to support that for the forseeable future.
m
@schrockn, I doesn't bother me to rework all the repos to locations. I like the idea of locations as it more clear in terms of isolation. What actually bothers me: • Each location requires a separate section in deployments helm chart, which may be a significant amount of boilerplate even if you have reasonable amount of locations (at the moment we just use mono-repo with multiple repositories and that's easy to manage/upgrade) • Repositories help us to group jobs by some criteria in dagit interface. But if we add all of jobs to a single location then we will not have grouping mechanism for jobs (as I know it exists only for assets).
s
@Mykola Palamarchuk thanks for the feedback and understanding. I just approved a PR moving
create_repository_using_definitions_args
to a top-level include, so you’ll be able to import it from top-level after next week’s release.