19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
It looks like your image didn’t install the “celer...
# ask-communityd
daniel
12/27/2022, 1:02 PMIt looks like your image didn’t install the “celery” package. Is it possible to share your Dockerfile?
v
Vinu Praveen
12/27/2022, 1:19 PMI also use another dockerfile for user code. Will share that too.
d
daniel
12/27/2022, 1:40 PMCan you paste the text of the user code dockerfile rather than a screenshot?
daniel
12/27/2022, 1:41 PM(or as a file)
daniel
12/27/2022, 1:43 PMLooks like this is an issue with python 3.7 and the celery library, there are some workarounds suggested here https://github.com/celery/celery/issues/7783
daniel
12/27/2022, 1:46 PMlooks like the simplest fix is adding importlib-metadata<5.0 to the list of installed packages
v
Vinu Praveen
12/27/2022, 1:50 PMHi Daniel, thanks for the response. i am not sure if i can share you the file or the content as im working on my office laptop...
Vinu Praveen
12/27/2022, 1:52 PMbut i can try adding importlib-metadata<5.0, but could you suggest where should i add this?
Is it in the user code dockerfile? like in the below?
RUN pip install \
dagster==${DAGSTER_VERSION} \
dagster-postgres==${DAGSTER_VERSION} \
dagster-aws==${DAGSTER_VERSION} \
dagster-k8s==${DAGSTER_VERSION} \
dagster-celery[flower,redis,kubernetes]==${DAGSTER_VERSION} \
dagster-celery-k8s==${DAGSTER_VERSION} \
importlib-metadata<5.0
d
daniel
12/27/2022, 2:25 PMThat’s where I would add it yeah
v
Vinu Praveen
12/27/2022, 2:26 PMSure, thanks I'll try that. ☺️
Vinu Praveen
12/27/2022, 2:44 PM##[error]The command '/bin/sh -c pip install dagster==${DAGSTER_VERSION} dagster-postgres==${DAGSTER_VERSION} dagster-aws==${DAGSTER_VERSION} dagster-k8s==${DAGSTER_VERSION} dagster-celery[flower,redis,kubernetes]==${DAGSTER_VERSION} dagster-celery-k8s==${DAGSTER_VERSION} importlib-metadata<5.0' returned a non-zero code: 2
##[error]The process '/usr/bin/docker' failed with exit code 2
guess this should be some syntax error.
d
daniel
12/27/2022, 3:18 PMtry
importlib-metadata\<5.0v
Vinu Praveen
12/27/2022, 3:43 PMsure..
Vinu Praveen
12/27/2022, 3:56 PMawesome, it worked !
🎉 1
Vinu Praveen
12/27/2022, 4:09 PMAnd i have another question. 🙂 i have a requirement to have the user code deployment as separate and to have daemon/dagit deployment to be separate.
I did read an article from dagster where i will have to madify the below in the dagster values.yaml file.
workspace:
enabled: true
enableSubchart: false.
and after making the above changes it also says the dagster-user-deployments will be managed on its own.
But not sure how will it be manages and what else i will have to modify for Separately Deploying Dagster Infrastructure and User Code.
https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment
d
daniel
12/27/2022, 4:09 PMCan you say more what you mean by "how will it be managed"?
v
Vinu Praveen
12/27/2022, 4:10 PMthis is the line from the article itself
"Finally, the
dagster-user-deploymentsd
daniel
12/27/2022, 4:10 PMYou still have to helm upgrade that chart when you make changes to your code
daniel
12/27/2022, 4:11 PMbut you no longer have to upgrade the release with dagit/the daemon/ etc.
v
Vinu Praveen
12/27/2022, 4:12 PMcan we have a quick call if possible? 🙂
d
daniel
12/27/2022, 4:13 PMI can't do a call, but we can answer questions here
v
Vinu Praveen
12/27/2022, 4:17 PMsure thats fine. now i see i have 2 charts,
1. dagster
2. dagster-user-deployments
both has its own value.yaml files. now for separately deploying dagster infrastructure and user code i make the below changes in dagster(values.yaml)
workspace:
enabled: true
enableSubchart: false
then the user code deployment from openshift disappears. so this clearly means this it wants me to have user code deployment separate. am i correct?
Vinu Praveen
12/27/2022, 4:18 PMim sorry, im very new to this 🙂
d
daniel
12/27/2022, 4:19 PMno problem - you can choose to either do everything in a single helm chart and helm release, or split things out into two helm charts and two releases as described here: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#separately-deploying-dagster-infrastructure-and-user-code
The single chart approach is probably simpler if you want to keep it simple. If you do use two separate helm charts, you only need to helm update the dagster-user-deployments helm chart when your code changes
v
Vinu Praveen
12/27/2022, 4:22 PMalright, so i can simply deploy the dagster-user-deployment chart like i did for dagster chart. using the dagster-user-deployment(values.yaml) file.
and i guess it will create another helm release in the opnshift side beside the already deployed dagster helm release.
d
daniel
12/27/2022, 4:23 PMI believe that's correct, yeah
daniel
12/27/2022, 4:23 PMTwo separate helm releases
v
Vinu Praveen
12/27/2022, 4:24 PMok, so thats clear now, what if i want only one helm release only in openshift but i also want to separately deploy dagster infrastructure and user code.. is that possible?
d
daniel
12/27/2022, 4:24 PMI don't think that is possible, no
v
Vinu Praveen
12/27/2022, 4:26 PMok fine. so if i want to separately deploy the user code it will be another helm release. got it.. I will work on the separate deployment part and will get back.. 🙂
thanks for all your assistance..
condagster 1
2 Views