The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi folks. I'm trying to setup a celery worker. I have rabbit mq with a hostname of rabbitmq. I've set the executor celery url in the $DAGSTER_HOME/pipeline_run.yaml and celery_config.yml as follows:

config:
-&gt; broker: '<pyamqp://guest>:guest@rabbitmq:5672//'

starting the broker:
dagster-celery worker start -y /opt/dagster/dagster_home/celery_config.yaml

However my worker is still looking at localhost and failing to connect :thinking_face:

dagster_worker1_1   | [2020-06-28 10:37:24,256: ERROR/MainProcess] consumer: Cannot connect to <amqp://guest>:**@127.0.0.1:5672//: [Errno 111] Connection refused.

Am I missing something somewhere?

I it not clear from docs but there should be
`execution:`
  `celery:`
    `broker: '<pyamqp://guest>:guest@rabbitmq:5672//'`

<https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-celery/dagster_celery/cli.py#L32>

Screenshot 2020-06-28 at 15.03.48.png

this is what I got, I'll try with your suggestion - take out config

Top banana... I took the docs literally where hints that it has to be the same as the pipeline_run.yaml

Which looks like this:

execution:
  celery:
    ~*config*~:
      broker: '<pyamqp://guest>:guest@rabbitmq:5672//'

Can't argue with the code! Thank you!