https://dagster.io/ logo
#ask-community
Title
# ask-community
d

daniel

12/27/2022, 1:02 PM
It looks like your image didn’t install the “celery” package. Is it possible to share your Dockerfile?
v

Vinu Praveen

12/27/2022, 1:19 PM
I also use another dockerfile for user code. Will share that too.
d

daniel

12/27/2022, 1:40 PM
Can you paste the text of the user code dockerfile rather than a screenshot?
(or as a file)
Looks like this is an issue with python 3.7 and the celery library, there are some workarounds suggested here https://github.com/celery/celery/issues/7783
looks like the simplest fix is adding importlib-metadata<5.0 to the list of installed packages
v

Vinu Praveen

12/27/2022, 1:50 PM
Hi Daniel, thanks for the response. i am not sure if i can share you the file or the content as im working on my office laptop...
but i can try adding importlib-metadata<5.0, but could you suggest where should i add this? Is it in the user code dockerfile? like in the below? RUN pip install \ dagster==${DAGSTER_VERSION} \ dagster-postgres==${DAGSTER_VERSION} \ dagster-aws==${DAGSTER_VERSION} \ dagster-k8s==${DAGSTER_VERSION} \ dagster-celery[flower,redis,kubernetes]==${DAGSTER_VERSION} \ dagster-celery-k8s==${DAGSTER_VERSION} \ importlib-metadata<5.0
d

daniel

12/27/2022, 2:25 PM
That’s where I would add it yeah
v

Vinu Praveen

12/27/2022, 2:26 PM
Sure, thanks I'll try that. ☺️
##[error]The command '/bin/sh -c pip install dagster==${DAGSTER_VERSION} dagster-postgres==${DAGSTER_VERSION} dagster-aws==${DAGSTER_VERSION} dagster-k8s==${DAGSTER_VERSION} dagster-celery[flower,redis,kubernetes]==${DAGSTER_VERSION} dagster-celery-k8s==${DAGSTER_VERSION} importlib-metadata<5.0' returned a non-zero code: 2 ##[error]The process '/usr/bin/docker' failed with exit code 2 guess this should be some syntax error.
d

daniel

12/27/2022, 3:18 PM
try
importlib-metadata\<5.0
(escaping)
v

Vinu Praveen

12/27/2022, 3:43 PM
sure..
awesome, it worked !
🎉 1
And i have another question. 🙂 i have a requirement to have the user code deployment as separate and to have daemon/dagit deployment to be separate. I did read an article from dagster where i will have to madify the below in the dagster values.yaml file. workspace: enabled: true enableSubchart: false. and after making the above changes it also says the dagster-user-deployments will be managed on its own. But not sure how will it be manages and what else i will have to modify for Separately Deploying Dagster Infrastructure and User Code. https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment
d

daniel

12/27/2022, 4:09 PM
Can you say more what you mean by "how will it be managed"?
v

Vinu Praveen

12/27/2022, 4:10 PM
this is the line from the article itself "Finally, the
dagster-user-deployments
subchart can now be managed in its own release."
d

daniel

12/27/2022, 4:10 PM
You still have to helm upgrade that chart when you make changes to your code
but you no longer have to upgrade the release with dagit/the daemon/ etc.
v

Vinu Praveen

12/27/2022, 4:12 PM
can we have a quick call if possible? 🙂
d

daniel

12/27/2022, 4:13 PM
I can't do a call, but we can answer questions here
v

Vinu Praveen

12/27/2022, 4:17 PM
sure thats fine. now i see i have 2 charts, 1. dagster 2. dagster-user-deployments both has its own value.yaml files. now for separately deploying dagster infrastructure and user code i make the below changes in dagster(values.yaml) workspace: enabled: true enableSubchart: false then the user code deployment from openshift disappears. so this clearly means this it wants me to have user code deployment separate. am i correct?
im sorry, im very new to this 🙂
d

daniel

12/27/2022, 4:19 PM
no problem - you can choose to either do everything in a single helm chart and helm release, or split things out into two helm charts and two releases as described here: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#separately-deploying-dagster-infrastructure-and-user-code The single chart approach is probably simpler if you want to keep it simple. If you do use two separate helm charts, you only need to helm update the dagster-user-deployments helm chart when your code changes
v

Vinu Praveen

12/27/2022, 4:22 PM
alright, so i can simply deploy the dagster-user-deployment chart like i did for dagster chart. using the dagster-user-deployment(values.yaml) file. and i guess it will create another helm release in the opnshift side beside the already deployed dagster helm release.
d

daniel

12/27/2022, 4:23 PM
I believe that's correct, yeah
Two separate helm releases
v

Vinu Praveen

12/27/2022, 4:24 PM
ok, so thats clear now, what if i want only one helm release only in openshift but i also want to separately deploy dagster infrastructure and user code.. is that possible?
d

daniel

12/27/2022, 4:24 PM
I don't think that is possible, no
v

Vinu Praveen

12/27/2022, 4:26 PM
ok fine. so if i want to separately deploy the user code it will be another helm release. got it.. I will work on the separate deployment part and will get back.. 🙂 thanks for all your assistance..
condagster 1
2 Views