I have a strange issue in dagster and dagster-celery. In my local environment, everything works fine as expected.
However, In prod in AWS ECS, sometimes, a task is submitted but i do not see any task in a task queue. in dagit UI event log I can see that the task was submitted to the queue. Then, job get stuck there.
Did anyone have the same issue as this? Or if anyone knows about this issue, can you point me how to fix this issue?
FYI, dataster = 0.15.2 ann dagster-celery = 0.15.2
01/11/2023, 4:26 PM
do you have flower deployed as well? that will give you better insight into what’s happening with your celery queues