https://dagster.io/ logo
#ask-community
Title
# ask-community
f

Flavien

02/09/2023, 2:25 PM
Hello 👋 I'm trying to run Dagit in an Azure Container App. Everything is fine except that runs appear or disappear when definitions are reloaded. Sometimes, browsing a specific run results in a message saying The run with this ID does not exist or has been cleaned up and sometimes not. I turned on logging at DEBUG level and I see somes errors related to a websocket connection:
Copy code
2023-02-09T14:03:21.876889299Z DEBUG:    = connection is CONNECTING
2023-02-09T14:03:21.877749315Z DEBUG:    < GET /graphql HTTP/1.1

2023-02-09T14:03:21.878301025Z DEBUG:    < cache-control: no-cache
2023-02-09T14:03:21.878666532Z DEBUG:    < pragma: no-cache
2023-02-09T14:03:21.878680732Z DEBUG:    < accept-encoding: gzip, deflate, br
2023-02-09T14:03:21.878685632Z DEBUG:    < accept-language: en-US, en; q=0.9
2023-02-09T14:03:21.878691232Z DEBUG:    < cookie: AppServiceAuthSession=XXX
2023-02-09T14:03:21.878697432Z DEBUG:    < host: <http://XXX.azurecontainerapps.io|XXX.azurecontainerapps.io>
2023-02-09T14:03:21.878702632Z DEBUG:    < user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.78
2023-02-09T14:03:21.878707433Z DEBUG:    < origin: <https://XXX.azurecontainerapps.io>
2023-02-09T14:03:21.878711933Z DEBUG:    < sec-websocket-version: 13, 13
2023-02-09T14:03:21.878716633Z DEBUG:    < sec-websocket-key: XXX, YYY
2023-02-09T14:03:21.878730733Z DEBUG:    < sec-websocket-extensions: permessage-deflate; client_max_window_bits
2023-02-09T14:03:21.878735033Z DEBUG:    < sec-websocket-protocol: graphql-ws, graphql-ws
2023-02-09T14:03:21.878747533Z DEBUG:    < x-forwarded-for: XXX
2023-02-09T14:03:21.878755233Z DEBUG:    < x-envoy-external-address: XXX
2023-02-09T14:03:21.878760234Z DEBUG:    < x-request-id: XXX
2023-02-09T14:03:21.878765634Z DEBUG:    < x-envoy-expected-rq-timeout-ms: 1800000
2023-02-09T14:03:21.878770134Z DEBUG:    < x-k8se-app-name: XXX
2023-02-09T14:03:21.878775534Z DEBUG:    < x-k8se-app-namespace: k8se-apps
2023-02-09T14:03:21.878779734Z DEBUG:    < x-k8se-protocol: http1
2023-02-09T14:03:21.878784034Z DEBUG:    < x-k8se-app-kind: web
2023-02-09T14:03:21.878788834Z DEBUG:    < x-ms-containerapp-name: XXX
2023-02-09T14:03:21.878793034Z DEBUG:    < x-ms-containerapp-revision-name: XXX
2023-02-09T14:03:21.879612149Z DEBUG:    < x-arr-ssl: true
2023-02-09T14:03:21.879624450Z DEBUG:    < x-forwarded-proto: https
2023-02-09T14:03:21.879629250Z DEBUG:    < x-ms-client-principal-name: XXX
2023-02-09T14:03:21.879633650Z DEBUG:    < x-ms-client-principal-id: XXX
2023-02-09T14:03:21.879638050Z DEBUG:    < x-ms-client-principal-idp: XXX
2023-02-09T14:03:21.879653750Z DEBUG:    < x-ms-client-principal: XXX
2023-02-09T14:03:21.879674351Z DEBUG:    < connection: Upgrade
2023-02-09T14:03:21.879678851Z DEBUG:    < upgrade: websocket
2023-02-09T14:03:21.879683151Z DEBUG:    ! invalid handshake
2023-02-09T14:03:21.879687551Z Traceback (most recent call last):
2023-02-09T14:03:21.879691751Z   File "/usr/local/lib/python3.10/site-packages/websockets/legacy/handshake.py", line 85, in check_request
2023-02-09T14:03:21.879696551Z     raw_key = base64.b64decode(s_w_key.encode(), validate=True)
2023-02-09T14:03:21.879700751Z   File "/usr/local/lib/python3.10/base64.py", line 86, in b64decode
2023-02-09T14:03:21.879705151Z     raise binascii.Error('Non-base64 digit found')
2023-02-09T14:03:21.879709551Z binascii.Error: Non-base64 digit found
2023-02-09T14:03:21.879713651Z 
2023-02-09T14:03:21.879718051Z The above exception was the direct cause of the following exception:
2023-02-09T14:03:21.879722252Z 
2023-02-09T14:03:21.879726752Z Traceback (most recent call last):
2023-02-09T14:03:21.879731152Z   File "/usr/local/lib/python3.10/site-packages/websockets/legacy/server.py", line 165, in handler
2023-02-09T14:03:21.879735552Z     await self.handshake(
2023-02-09T14:03:21.879739852Z   File "/usr/local/lib/python3.10/site-packages/websockets/legacy/server.py", line 593, in handshake
2023-02-09T14:03:21.879748952Z     early_response = await early_response_awaitable
2023-02-09T14:03:21.879757552Z   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 166, in process_request
2023-02-09T14:03:21.879769852Z     websockets.legacy.handshake.check_request(headers)
2023-02-09T14:03:21.879774452Z   File "/usr/local/lib/python3.10/site-packages/websockets/legacy/handshake.py", line 87, in check_request
2023-02-09T14:03:21.879779153Z     raise InvalidHeaderValue("Sec-WebSocket-Key", s_w_key) from exc
2023-02-09T14:03:21.879783453Z websockets.exceptions.InvalidHeaderValue: invalid Sec-WebSocket-Key header: XXX, YYY
2023-02-09T14:03:21.879846654Z DEBUG:    > HTTP/1.1 400 Bad Request
2023-02-09T14:03:21.879852754Z DEBUG:    > Date: Thu, 09 Feb 2023 14:03:21 GMT
2023-02-09T14:03:21.879923155Z DEBUG:    > Content-Length: 125
2023-02-09T14:03:21.879970956Z DEBUG:    > Content-Type: text/plain
2023-02-09T14:03:21.880003157Z DEBUG:    > Connection: close
2023-02-09T14:03:21.880063458Z DEBUG:    > [body] (125 bytes)
2023-02-09T14:03:21.880153060Z INFO:     connection failed (400 Bad Request)
2023-02-09T14:03:21.880185960Z DEBUG:    x closing TCP connection
2023-02-09T14:03:21.882107496Z DEBUG:    = connection is CLOSED
2023-02-09T14:03:21.882126396Z INFO:     connection closed
The container app is protected by Azure AD authentication. I see a related error in the Azure Container App runtime logs:
Copy code
Middleware[0] Failed to forward request to <http://127.0.0.1:3000>. Encountered a System.Net.WebSockets.WebSocketException exception after 7.038ms with message: The server returned status code '400' when status code '101' was expected.. Check application logs to verify the application is properly handling HTTP traffic.
Do you have any clue on what's going on?
4 Views