Hi I d like to use asyncpg for my postgres connection from t

Question

Hi I d like to use asyncpg for my postgres connection from the dagster daemon to the DB I use the official HELM charts Any idea how I can inject the asyncpg python package into the dagster daemon Docker Pod I don t see an option in the values yaml Basically I d liek to do the following `postgresqlScheme = postgresql+asyncpg ` in the values yaml The reason I d like to try this is that the dagster daemon reconnects to our postgres server every second causing a lot of cpu load Our other connections work just fine How do I increase the log level of the dagster daemon itself Not the Compute Logs or anything I mean the dasgter daemon process pod

rex · Answer

I believe we would need to refactor the daemon to be async compatible for you to use this option I don t think you can get the benefits of the package without the refactor Feel free to file an feature request for this transitioning to async await is definitely a big undertaking

Timo · Answer

ok maybe I don t even need that but I can t debug the reason why the daemon constantly reconnects to my postgresserver The PG logs tell me dagster daemon unexpectedly closes the connection shortly after being successfully established How can I increase the log level of the dagster daemon process in combination with your helm chart

Timo · Answer

You have hardcoded the loglevel to INFO of the daemon in ` utils logs py and daemon controller py`

Timo · Answer

< rex> I think this is worth an issue on github I guess I won t be the last who has to debug connection related issues The loglevel for the daemon should be adjustable in the HELM s values yaml eventually

Timo · Answer

I can set an postgresql scheme in the `values yaml` but even the example does not works because pg8000 is not installed in the dagster daemon pod ``` When set overrides the default `postgresql` scheme for the connection string e g set to `postgresql+pg8000` to use the `pg8000` library See <https docs sqlalchemy org en 13 dialects postgresql html dialect postgresql> postgresqlScheme ``` ```2022 08 25T09 35 41 608289872Z ModuleNotFoundError No module named pg8000 ```