Hello, just wondering if there's a best practice or recommendation for storing credentials for use in pipelines, e.g. a key for API calls executed with dagster? I tried searching, but only found a few things related to specific services. Thanks!

I have solved this by creating a resource which stored credentials. All ops that need credentials depend on the credentials resource. I have then created a credentials resource which accesses Google Cloud Secret Manager using authenticated API calls (authenticated using a local service account key file). One (secure) place to manage all credentials. I have also created a credentials resource with the same interface is the one mentioned above, but can be populated with a

dict

- this is useful for testing and for development (rather than having to query Secret Manager every time). Using Secret Manager is an implementation detail - use any means for storing and accessing your credentials. But have the credentials abstracted away as a resource works really well.