https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jason

08/15/2022, 4:16 PM
Hello... regarding
snowflake_io_manager
is the expectation that these (asset) tables already exist in Snowflake making them somewhat "strongly typed" to your dataframe (Software Defined Asset) in code? Or does the io manager wrap a
create table
statement around things similar to how dbt works? (for me it only works when tables are already created so just confirming/validating if this is the expected behaviour)
🤖 1
s

sandy

08/15/2022, 4:36 PM
Hey Jason - the current version doesn't create the table, but it probably should. @jamie - is this something that you'd have the bandwidth to add?
j

Jason

08/15/2022, 4:43 PM
That would be great - currently I'm using
snowflake_resource
in the asset to create (if not exist) the table first, then do stuff, then
snowflake_io_manager
handles it
j

jamie

08/15/2022, 4:53 PM
hi @Jason what version of dagster are you using? I'm pretty sure I added this capability in a couple weeks ago, and confirmed that tables get created for me when i materialize a table to snowflake for the first time (but maybe there's something different about my setup). if you have any error messages you can share that would also help!
j

Jason

08/15/2022, 5:03 PM
Originally, I was on
0.15.6
but last week went to
1.0
and I haven't tried since bc the code was already setup and running in my branch. In a couple hours I'll comment out my code, drop the existing table and re-try. Then get back to you here.
j

jamie

08/15/2022, 5:19 PM
great! my guess is it will be fixed with the upgrade, i fixed a couple other things for the snowflake io manager in 0.15.8 and i think that was one of them
j

Jason

08/15/2022, 8:04 PM
@jamie Fixed! This is great thank you. I'll re-test some other stuff now that I'm on
v1
before reporting any potential bugs
As a side note, I believe you're using
fetchall()
for
snowflake_resource.execute_query()
but it would be nice to also have the option to return data in pandas dataFrame with
fetch_pandas_all()
j

jamie

08/15/2022, 8:24 PM
glad it's fixed! looks like you can do
snowflake_resource.execute_query(query, use_pandas_result=True)
to get it back as a df!
j

Jason

08/15/2022, 8:29 PM
Thanks @jamie
j

jamie

08/15/2022, 8:30 PM
oh haha, didn't realize it was that new! you might need a 1.0.3 upgrade then
j

Jason

08/15/2022, 8:44 PM
Nope, just checked. It barely missed out of
1.0.3
so it will show up in
1.0.4
on Thursday. I can wait 🙂