Is there any way to wrap the

snowflake_resource

in an sqlalchemy connection? Or would you guys be open to a PR to make the

snowflake_resource

use sqlalchemy? The reason is, some pandas function such as

.to_sql

require an sqlalchemy connection, so as far as I can tell / and via my own testing the pandas SQL utilities won't be available for Snowflake using the Dagster

snowflake_resource

directly as it is now. Any thoughts?

Not sure if you've seen this already, but the snowflake python package

pandas_tools

module also has a builtin

write_pandas()

method for doing this: https://docs.snowflake.com/en/user-guide/python-connector-api.html#label-python-connector-api-write-pandas

Yeah that all makes sense and I was aware of the

write_pandas

function. The reason I've been wanting to use

.to_sql

from my dataframe is because it will create the table for me if it doesn't exist already, with the proper schema. I'm migrating a postgres DB to snowflake and have 50+ tables to copy over, so I don't want to have to manually create each table, define types etc. Any other workaround you can think for this issue?

ah I see I see. Yeah, not sure of another route that would create the tables for you. The only thing I could think of is to create a solid that iterates through all postgres tables, inspects the schema, and dynamically creates and runs an equivalent

CREATE TABLE

snowflake command before copying data

@Basil V I've actually faced this issue myself. If you wanted to submit a PR, I'd be happy to take a look

@sandy just put up a PR https://github.com/dagster-io/dagster/pull/3214 — I tried to maintain all the default behavior, very open to suggestions for improving it. Lmk what you guys think. Thanks!

Thanks @Basil V! I'll take another look

Hey @sandy I went through your second review comments, updated the formatting etc. It looks like it is still failing 3 tests—but not sure if those are due to my PR or something else. Anyways let me know if it looks good or if you need me to make any other changes. Thanks again!

Hey Basil - our next release should be on Thursday this week

Awesome thanks!