06/24/2022, 6:31 PM
Upon further reading, would it make sense to wrap SQLAlchemy model classes with an IOManager class?
06/24/2022, 7:17 PM
If your ops/assets are producing objects that you want to be serialized to postgres and then deserialized from postgres for downstream ops, then an IO manager would be a good choice
06/24/2022, 8:00 PM
Would IO Managers still make sense if I wish to store the final results of my ops in a postgres db (as opposed to say snowflake)
Trying to configure SQLAlchemy with an IOManager to write directly to postgres for final results. Currently pickling and writing to S3 for intermediary steps.
06/24/2022, 8:01 PM
yeah - from Dagster's perspective, there's no difference between snowflake and postgres
you can use different IO managers at different points in your job / for different assets. some can go to S3 and others can go to postgres