https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jayme Edwards

03/27/2023, 3:11 PM
Hey y'all, I recently switched from using a repository to definitions for setting up all my dagster assets, sensors, and jobs. So far so good running locally. However I deploy to kubernetes in our staging environment (this project isn't in production quite yet). There used to be a workspace.yml file that specified the port that was used for communication between dagit and dagster and this is removed when using defintions. Is there anything I need to change to make sure those two can communicate in k8? Or is that already handled since I'm using the helm chart? In the values.yaml for helm there's a section that looks like this I've been using during deployment to get the usercode stuff to talk to dagster/dagit:
dagsterApiGrpcArgs:
- "--python-file" - "hq_dagster/repository.py" - "--working-directory" - "/opt/dagster/app" - "-h" - "0.0.0.0"
🤖 1
d

daniel

03/27/2023, 3:13 PM
Hi Jayme - no changes should be required here. (There's actually still a workspace.yaml under the hood here, but it's entirely managed by the Helm chart)
j

Jayme Edwards

03/27/2023, 3:25 PM
Thanks @daniel in my config code above there's a reference to the repository.py which no longer exists. What would you recommend I change with that? Appreciate your help as always.
d

daniel

03/27/2023, 3:26 PM
Is there a new file or module that you can reference instead?
j

Jayme Edwards

03/27/2023, 3:49 PM
I followed some examples in the issues for when the repository to definitions change was made. This recommended putting the definitions file in a module initialization file containing the usercode assets of the source. In my case
hq_dagster/__init__.py
d

daniel

03/27/2023, 3:49 PM
When you run this code locally, what command do you run?
(in
dagster dev
or
dagit
)
j

Jayme Edwards

03/27/2023, 3:50 PM
dagster dev -p 9000
d

daniel

03/27/2023, 3:50 PM
What's in your pyproject.toml?
j

Jayme Edwards

03/27/2023, 3:50 PM
[build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [tool.dagster] module_name = "hq_dagster"
d

daniel

03/27/2023, 3:50 PM
This should work then:
Copy code
dagsterApiGrpcArgs:
          - "--module-name"
          - "hq_dagster"
          - "--working-directory"
          - "/opt/dagster/app"
          - "-h"
          - "0.0.0.0"
j

Jayme Edwards

03/27/2023, 3:52 PM
Sweet. Thanks man! I'm sure y'all are planning it, but maybe a doc topic on how to convert from repositories to definitions in the helm/deployment area would help here. Not sure how many people are doing k8 though. Python is one of the languages I have less experience with so some of this stuff may be elementary to more people though.
d

daniel

03/27/2023, 3:59 PM
absolutely, i can pass that along to our docs hit list
❤️ 1
b

Binoy Shah

03/27/2023, 4:13 PM
I am also using Helm, but I have multi module repo, is it possible to pass in multiple
--module-name
arguments to the
dagsterApiGrpcArgs
?
d

daniel

03/27/2023, 4:16 PM
It's not currently possible to specify multiple modules to a gRPC server - that's a limitation of the underlying command, not the Helm chart
What we recommend there currently is to have a single entrypoint module that imports the repositories from each of the modules
b

Binoy Shah

03/27/2023, 4:18 PM
right that’s what I am doing but that would mean I have to keep using repositories and not use the new
Definitions
d

daniel

03/27/2023, 4:18 PM
Oh i see - right now you can only have one Definitions object per code location. If you want multiple repositories, our recommend is to keep using repository, there are no plans to remove it
If you want to use the new Definitions syntax with a repository, you can use the
create_repository_using_definitions_args
function
b

Binoy Shah

03/27/2023, 4:20 PM
oh.. interesting.. yes that might be worthwhile switching to it
So this link https://docs.dagster.io/_apidocs/definitions#dagster.create_repository_using_definitions_args still says its a “stopgap” solution, is there anything better already in the works ?
d

daniel

03/27/2023, 4:22 PM
I don't think currently - I expect that to work for quite some time
😃 1
👍 1
31 Views