https://dagster.io/ logo
Powered by
c

Chris Roth

04/02/2020, 8:41 PM
hey, i'm running 
dagster-celery worker start -y celery_config.yaml
with an sqs broker and i get 
ValueError: invalid literal for int() with base 10:
. i think the solution is to safequote these strings (eg https://docs.celeryproject.org/en/latest/getting-started/brokers/sqs.html), but i'm not sure if there's a way to do this without forking dagster-celery. any ideas?
a

alex

04/02/2020, 8:44 PM
whats the change to make to dagster-celery? we could get the fix in today
c

Chris Roth

04/02/2020, 8:46 PM
actually - it might be possible to fix this by url encoding the password first - let me try this
yeah, that seems to have fixed it
in theory the fix would be exactly what the celery doc does with 
safequote
or just url encoding the pw
a

alex

04/02/2020, 8:47 PM
k its not clear to me if we should try to handle any of that at our level
since we dont know whats in the broker_url and don’t want to incorrectly munge it
but if you’re understanding of the world is different let me know
c

Chris Roth

04/02/2020, 8:48 PM
yeah i'm not sure either. it would be cool if there was a page in the docs about how to set up SQS and i think simply mentioning that the pw needs to be url encoded would be sufficient
but definitely don't think it's really a dagster specific issue
i think the tricky thing is that since dagster gets config from yaml, it inhibits the developer's ability to do safequote or urlencode in python. but for now i think this is a non issue then
a

alex

04/02/2020, 8:53 PM
is this sufficient https://dagster.phacility.com/D2419 or how would you word it?
c

Chris Roth

04/02/2020, 8:54 PM
i think that makes sense - maybe even link to the github issue?
Powered by