@Scout when I dump a dataframe into Snowflake, is it by default an overwrite ? Can I make it so data is appended instead ?

When using Dagster to write data to Snowflake, the behavior of whether data is overwritten or appended depends on the configuration of the I/O manager that you are using. According to the documentation, if you use the default Snowflake I/O manager provided by Dagster, it will by default delete the data from the selected partition and then insert the content of your dataframe. This ensures that you only have fresh data in your table, which implies an overwrite behavior. If you want to append data instead of overwriting it, you would need to configure the I/O manager accordingly. The documentation does not provide a direct example of configuring the Snowflake I/O manager to append data, but you can typically control this behavior by customizing the I/O manager's

handle_output

method. If you need to append data and the default behavior of the provided Snowflake I/O manager does not meet your requirements, you may consider writing a custom I/O manager that inherits from

ConfigurableIOManager

and implements the desired behavior in the

handle_output

method. For specific code examples and configurations, you should refer to the Dagster documentation or the source code of the Snowflake I/O manager to understand how to customize or extend it to support appending data instead of overwriting. Clean example of a partitioned asset wit... button Dagster Docs button Testing against production with Dagster ... button dagster-snowflake integration reference button Thank you for your feedback!