Figuring out how to deploy Dagster for my companie...
# ask-community
l
Figuring out how to deploy Dagster for my companies environment (GKE). And I'm not sure exactly how I'm supposed to containerize and deploy my "user code". This guide explains decoupling the code base, and I'm assuming I need to modify The helm chart of
dagster/dagster-user-deployments
needs to be modifed
Copy code
# List of unique deployments
  deployments:
    - name: "k8s-example-user-code-1"
      image:
        # When a tag is not supplied, it will default as the Helm chart version.
        repository: "<http://docker.io/dagster/user-code-example|docker.io/dagster/user-code-example>"
        tag: ~

        # Change with caution! If you're using a fixed tag for pipeline run images, changing the
        # image pull policy to anything other than "Always" will use a cached/stale image, which is
        # almost certainly not what you want.
        pullPolicy: Always
Such that the
repository
is pointing to an image of my user code. I'm trying to find the source code for https://hub.docker.com/r/dagster/user-code-example or any explanation for how this is supposed to be structured
dagster bot responded by community 1
The helm chart will set the proper command and entrypoint for your user code, provided that you pass the matching values to the chart for the python module/file to load
l
Sick
Giving this a shot, thank you
I'm unable to get my user-code helm chart to update to the new image. Per the documents in the dagster chart I set the enableSubchart to false. In the user-deployment helm chart I modified
Copy code
deployments:
    - name: "unifyco-dagster"
      image:
        # When a tag is not supplied, it will default as the Helm chart version.
        repository: PRIVATE_REPO_URL
        tag: "0.0.1"
        imagePullSecrets: PRIVATE_REPO_SECRET
Yet when I apply it's spinning up the user-repo pod from the demo.
Copy code
NAME                                                     READY   STATUS      RESTARTS   AGE
dagster-daemon-795899b4b8-mflph                          1/1     Running     0          4h1m
dagster-dagit-6fdf977b95-868z8                           1/1     Running     0          4h1m
dagster-postgresql-0                                     1/1     Running     0          9h
dagster-run-8ea28b97-46b9-48aa-9281-3b19194ecde3-86nh4   0/1     Completed   0          9h
lpieri@unifyai unify-dagster % helm upgrade --install user-code dagster/dagster-user-deployments -f helm/user-code-values.yaml
Release "user-code" does not exist. Installing it now.
NAME: user-code
LAST DEPLOYED: Mon Jul 10 22:02:43 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
lpieri@unifyai unify-dagster % k get pods                                                                                     
...
user-code-dagster-user-deployments-k8s-example-user-code-1lrcbh   1/1     Running     0          8s
That pod
user-code...
is the image I was using previously.
a
It sounds like something isn’t quite right with your values file if I had to guess. Is the snippet above the whole file? Can you share enough to see the overall structure?
l
The deployment contains the image despite it not being mentioned in the yaml.
a
ah ha - yeah, as I suspected, helm issue. you used a values.yaml for the main chart (i.e. https://github.com/dagster-io/dagster/blob/master/helm/dagster/values.yaml), rather than one for the user code subchart (https://github.com/dagster-io/dagster/blob/master/helm/dagster/charts/dagster-user-deployments/values.yaml). Since you didn't override these lines, the sample was deployed: https://github.com/dagster-io/dagster/blob/master/helm/dagster/charts/dagster-user-deployments/values.yaml#L25-L29
If you haven't worked much with helm before, I'd recommend stepping back and doing a tutorial there - it'll help you in the future. you can always check what was actually applied to the chart too
helm get values <helm-deployment-name>
l
will do a help tutorial and get back to this 🫡
blob salute
Thanks