Hi Guys! I am trying to test Celery execution in m...
# announcements
Hi Guys! I am trying to test Celery execution in my local environment based on your example (using rabbitmq). I try to start the pipeline from dagit playground with execution celery, starting, then stucking in some waiting....local celery and dagit were started from the same directory...and no exception, it is not logging anything...I put some debug stuff into the thread if there is someone who can check and give me some advice. thanks in advance
I try to go with this example:
from dagit I started like this
local celery got something
dagit log says submitting
continuously running / waiting unfortunatly I dont understand
no exception, no error just running...What is the problem here? 😉 - what do I miss?
I patched my code like it was in your example
Hmm could you try running ‘dagster-celery worker start -A dagster_celery.app -Q dagster’ to explicitly point the worker to the dagster queue? (Based on this https://github.com/celery/celery/issues/3759)
@cat thanks, tomorrow, I am going to check this...
thank you very much
Which also uses ‘-q dagster’
Yup lmk if that doesn’t work and I’ll keep digging
thanks, I will give you a feedback...
@cat Hi! So I did everything as you suggested but the behavior is just the same. I started celery (locally) like this: "dagster-celery worker start -A dagster_celery.app -q dagster --loglevel=INFO" and it is continuously running. maybe is it waiting for something?
@cat I have just read through the celery issue you mention, maybe this is my problem
I am going to try to deploy celery on kubernetes, and test there...
👍 1
Hey, just wanted to follow up and see if deploying celery on k8s went smoothly or if you ran into issues?