https://dagster.io/ logo
Powered by
g

Gaetan DELBART

03/17/2020, 6:50 AM
Hello Again ! I've worked on my k8s integration of dagster. After launching a 
helm install dagster helm/dagster
, my celery workers fail to start The result of a 
kubectl logs pod-name
:
Copy code
Traceback (most recent call last):
  File "/usr/local/bin/dagster-celery", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/dagster_celery/cli.py", line 159, in worker_start_command
    pythonpath = get_config_dir(config_yaml)
  File "/usr/local/lib/python3.7/site-packages/dagster_celery/cli.py", line 69, in get_config_dir
    if 'broker' in validated_config:
TypeError: argument of type 'NoneType' is not iterable
if I run a kubectl exec on the pod, and look at the content of the celery-config.yaml, it looks like this
Copy code
execution:
  celery:
      broker:  "<pyamqp://test>:test@dagster-rabbitmq:5672//"
      backend: 
      config_source:
So, it looks like the helm template is working as expected. Dos anyone has any advice about this ?
After digging around in the source code, I find this comment in https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-celery/dagster_celery/executor.py
Copy code
execution:
          celery:
            config:
              broker: '<pyamqp://guest>@localhost//'  # Optional[str]: The URL of the Celery broker
              backend: 'rpc://' # Optional[str]: The URL of the Celery results backend
              include: ['my_module'] # Optional[List[str]]: Modules every worker should import
              config_source: # Dict[str, Any]: Any additional parameters to pass to the
                  #...       # Celery workers. This dict will be passed as the `config_source`
                  #...       # argument of celery.Celery().
But, even if I change the helm template, I can't get the worker to start. (Same error as the one above)
Ok found it, I've found what went wrong ! The config can't have empty lines
Copy code
execution:
  celery:
      broker:  "<pyamqp://test>:test@dagster-rabbitmq:5672//"
n

nate

03/17/2020, 3:28 PM
ah thanks for flagging this @Gaetan DELBART, and for debugging the issue! I filed https://github.com/dagster-io/dagster/issues/2289 and will get this fixed shortly
❤️ 1
@Gaetan DELBART this is now fixed on master and will be in the next release—thanks again for the report!
2 Views
Powered by