https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Alan Dao

09/09/2022, 4:42 AM
do we have already written io manager for some common database or query engine already
o

Oliver

09/09/2022, 7:44 AM
also on the lookout for something like this I have seen in the dagster-snowflake code base their is a generic interface that is implemented by the snowflake io manager. seems like a good place to start. I wonder how if there's much standing in the way of an sqalchemy backend. That would hit a lot of db's with one implementation... interested to know if you find something existing though
d

Daniel Mosesson

09/09/2022, 11:28 AM
I wrote an IOManager that goes from
List[Dict]
+ optional metadata <-> Postgres. It covers insert, upsert, truncate + insert, pkeys and is still < 100 lines of code, so its really not hard
a

Alan Dao

09/09/2022, 12:53 PM
Its not about hard or not… i just want to have something to read and refer
y

yuhan

09/09/2022, 5:03 PM
in the fully featured project (docs), we have several io managers that interact with databases: https://github.com/dagster-io/dagster/tree/master/examples/project_fully_featured/project_fully_featured/resources
6 Views