19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Is there a good “dagster on kubernetes” example I ...
# deployment-kubernetesm
Matt Menzenski
07/19/2022, 9:39 PMIs there a good “dagster on kubernetes” example I can pull down and run in a test cluster?
r
rex
07/19/2022, 9:40 PMHave you checked out https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm?
m
Matt Menzenski
07/19/2022, 9:43 PMI’ve seen it, but am not familiar with helm. Is that the best example to try out?
r
rex
07/19/2022, 9:44 PMYeah that’s the best example to try out. Helm basically allows us to package a set of Kubernetes resources that define a Dagster deployment, as well as expose some useful, common configuration on those resources.
👍 1
m
Matt Menzenski
07/19/2022, 9:47 PMthe docs here reference https://hub.docker.com/r/dagster/user-code-example - is that image’s source available? (is it https://github.com/dagster-io/dagster/tree/master/examples/deploy_k8s/example_project ?)
r
rex
07/19/2022, 9:49 PMyeah, that’s correct
m
Matt Menzenski
07/19/2022, 9:49 PMcool, thank you
Matt Menzenski
08/02/2022, 8:54 PMok I’ve figured out Helm enough to stand up Dagster in our test cluster with the example user code repo.
I can see that the user code example repo is published as a docker image:
Copy code
Environment:
DAGSTER_CURRENT_IMAGE: <http://docker.io/dagster/user-code-example:0.15.8|docker.io/dagster/user-code-example:0.15.8>Matt Menzenski
08/03/2022, 3:10 AMSeems like I need to customize the dagster-user-deployment Helm chart to reference my own docker image? https://github.com/dagster-io/dagster/blob/master/helm/dagster/charts/dagster-user-deployments/values.yaml#L28 and my own arguments? https://github.com/dagster-io/dagster/blob/master/helm/dagster/charts/dagster-user-deployments/values.yaml#L42-L44
a
Andrea Giardini
08/03/2022, 8:58 AMYeah that's right. Otherwise you can treat is as a completely separate release and not include it in dagster's helm chart
m
Matt Menzenski
08/03/2022, 2:22 PMthanks - I will probably go that route as I’m not very familiar with Helm, I think it’d be faster.
Am I understanding correctly that in the
dagsterApiGrpcArgs Copy code
dagsterApiGrpcArgs:
- "-f"
- "/example_project/example_repo/repo.py"@repositorydagster new-project PROJECT_NAMEdagsterApiGrpcArgsPROJECT_NAME/repository.pya
Andrea Giardini
08/03/2022, 2:29 PMYeah that's right
keanu thanks 1
m
Matt Menzenski
08/03/2022, 3:19 PMAnd is this the right Dockerfile to build the image for my new user code repository? (from https://docs.dagster.io/deployment/guides/docker#multi-container-docker-deployment)
Copy code
FROM python:3.7-slim
# Checkout and install dagster libraries needed to run the gRPC server
# exposing your repository to dagit and dagster-daemon, and to load
# the DagsterInstance
RUN pip install \
dagster \
dagster-postgres \
dagster-docker
# Set $DAGSTER_HOME and copy dagster instance there
ENV DAGSTER_HOME=/opt/dagster/dagster_home
RUN mkdir -p $DAGSTER_HOME
COPY dagster.yaml $DAGSTER_HOME
# Add repository code
WORKDIR /opt/dagster/app
COPY repo.py /opt/dagster/app
# Run dagster gRPC server on port 4000
EXPOSE 4000
# Using CMD rather than ENTRYPOINT allows the command to be overridden in
# run launchers or executors to run other commands using this image
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "-f", "repo.py"]a
Andrea Giardini
08/03/2022, 3:40 PMYou should copy your whole project in the docker image, not only repo.py
m
Matt Menzenski
08/03/2022, 3:40 PMah, that’’s a good catch