https://dagster.io/ logo
#integration-snowflake
Title
# integration-snowflake
s

Simrun Basuita

07/05/2023, 8:22 AM
Hello, I'm using SnowflakeResource and would like to submit multiple statements in one transaction. There is a pattern for this in the Snowflake docs, using the
autocommit=False
argument to
snowflake.connector.connect
(https://docs.snowflake.com/developer-guide/python-connector/python-connector-example#using-context-manager-to-connect-and-control-transactions). How can I get a Snowflake connection with
autocommit=False
from
SnowflakeResource.get_connection()
?
🤖 1
j

jamie

07/05/2023, 3:27 PM
hey @Simrun Basuita the
SnowflakeResource
accepts an
autocommit
configuration that is forwarded to the snowflake connection https://docs.dagster.io/_apidocs/libraries/dagster-snowflake#dagster_snowflake.SnowflakeResource
s

Simrun Basuita

07/05/2023, 3:49 PM
@jamie can I set that config for just one asset?
I'm guessing it's something I can pass to @asset()? Will take a look
thanks
j

jamie

07/05/2023, 3:56 PM
the config set for a resource is used for all assets. But i think the
snowflake.connector.Connection
object has an
autocommit
method you could use
Copy code
@asset 
def my_asset(snowflake: SnowflakeResource):
   with snowflake.get_connection() as conn:
      conn.autocommit(False)
https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-api#object-connection
s

Simrun Basuita

07/05/2023, 5:50 PM
ah, nice! Thanks! 🙂
f

Félix Tremblay

09/28/2023, 11:50 AM
Hey @Simrun Basuita, have you tried to append multiple statements into a single string and send that for execution? That's what I'm usually doing. It can be significantly faster, since it avoids the connection overhead
s

Simrun Basuita

09/28/2023, 12:23 PM
Hey, I haven't! Thanks for the tip @Félix Tremblay
4 Views