Jacob Marcil

11/10/2022, 2:35 PM
Hello again 🙂 I have all my secrets in AWS Secrets Manager and want to configure as little as possible Github Secrets. So I’ve configured my AWS Credentials in Github Secrets so I can use
to gather anything I need from AWS Secrets. The problem I have is that
'add_to_environment': True,
puts all the retrieved json payload from AWS into a single environment variable with my secret key as the Env name. Then When I want to use
I’m not able to reference the json fields in my StringSource parameters.
"io_manager_snowflake": snowflake_io_manager.configured({
                    'account': {'env': 'SNOWFLAKE_ACCOUNT'},
                    'user': {'env': 'SNOWFLAKE_USER'},
                    'password': {'env': 'SNOWFLAKE_PASSWORD'},
                    'database': {'env': 'SNOWFLAKE_DATABASE'},
                    'schema': {'env': 'SNOWFLAKE_SCHEMA'},
                    'warehouse': {'env': 'SNOWFLAKE_WAREHOUSE'}
Is there a way to do that?
TLDR; Can you get json fields value with StringSource ?

Sean Lopp

11/10/2022, 4:20 PM
Hey Jacob, you could replace
with arbitrary code to parse the environment, but I think you're going to hit another problem, which is that the secretsmanager resource and the snowflake io manager are initialized at the same time, so the environment variables you're looking for most likely won't be set The AWS secrets resource is designed for jobs that need those secrets when they are run, not other resources that need them when the repo is initialized We're working on a more native approach to solve this problem, but for now here is a working example that uses a helper function to load the secrets into the environment when the repository is being initialized