https://dagster.io/ logo
#ask-community
Title
# ask-community
p

Paul Murray

05/30/2022, 8:34 PM
Hello, just wondering if there's a best practice or recommendation for storing credentials for use in pipelines, e.g. a key for API calls executed with dagster? I tried searching, but only found a few things related to specific services. Thanks!
🤖 1
s

Stefan Adelbert

05/31/2022, 1:55 AM
I have solved this by creating a resource which stored credentials. All ops that need credentials depend on the credentials resource. I have then created a credentials resource which accesses Google Cloud Secret Manager using authenticated API calls (authenticated using a local service account key file). One (secure) place to manage all credentials. I have also created a credentials resource with the same interface is the one mentioned above, but can be populated with a
dict
- this is useful for testing and for development (rather than having to query Secret Manager every time). Using Secret Manager is an implementation detail - use any means for storing and accessing your credentials. But have the credentials abstracted away as a resource works really well.
👍 2
11 Views