https://dagster.io/ logo
Title
n

Ness

04/26/2023, 9:34 PM
Hey folks, apologies if this is an elementary question: I am wanting to configure some assets to use an
IOManager
that reads/writes to a PostgreSQL table. Maybe I'm blind, but I haven't found an IOManager for PG specifically (I'm using dagster-postgres currently for my storage backend but it doesn't seem to have an IOManager). I've been looking at creating a new PostgreSQL
IOManager
based on the DuckDB one, but I realize it's enough work that I really ought to double-check that something like this doesn't already exist first. My current plan is to use the
dagster._core.storage.db_io_manager
with a custom
DbClient
modified to work with PostgreSQL, but I am not sure how much more work I'd have to do beyond that.
:rainbow-daggy: 2
c

chris

04/26/2023, 9:39 PM
not blind unfortunately - it doesn’t exist yet. I think it should be relatively straightforward once you have a custom db client and are able to successfully set up the connection
n

Ness

04/26/2023, 9:41 PM
Ok, good to hear. I'll keep plugging along. Any other tips besides creating my custom
DbClient
?
c

chris

04/26/2023, 9:41 PM
pinging lord of io managers @jamie for any thoughts here
n

Ness

04/27/2023, 12:18 AM
Penny for your thoughts, @jamie?
j

jamie

04/27/2023, 5:00 PM
Hey @Ness in addition to a DbClient, you’ll also need to write a DbTypeHanlder. here’s an example for duckdb and pandas https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-duckdb-pandas/dagster_duckdb_pandas/duckdb_pandas_type_handler.py
:dagster: 1
p

Philip Orlando

05/05/2023, 12:52 AM
FYI, I submitted a feature request related to this here.
:rainbow-daggy: 1