Is there a way to override Snowflake-materialized ...
# integration-snowflake
j
Is there a way to override Snowflake-materialized assets checking their
key_prefix
for schema information, and only use the schema value from the passed Snowflake IO manager? We run & test our pipelines in different schema configurations based on local dev, branch qa, and production, but ideally would like the visual organization of the assets in the UI to stay consistent between environments. So the goal would be to have static
key_prefixes
and dynamic io_manager schema configuration, that won't overlap in values
As a follow up, how exactly does a Snowflake materialized asset handle a compound key prefix. Our ideal organization pattern is
["db_name", "schema_name"]
as key prefixes. This makes aligning with key prefixes generated by DBT a lot simpler, and keeps our UI organized nicely. Does Dagster just attempt to load the asset into multiple schemas in this case?
j
hye @Joel Olazagasti totally hear you on handling schemas better and this in definitely on my mind as i start to work on some new snowflake features. As for having multiple values in the key prefix, in your case if the key prefix is
["db_name", "schema_name"]
then the io manager only picks the last key prefix. So in this case, it will store in
schema_name