https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
j

Josh Kutsko

06/02/2023, 6:06 PM
Is there a recommended way to set up google application credentials using dagster-cloud? Some of our jobs and assets need to authenticate, but not all of the resources have a credentials argument (e.g. GCSResource). We’ve set up gcp credentials in a dagster environment variable, and our current solution is to decode the base64 credentials key and set the GOOGLE_APPLICATION_CREDENTIALS file and env variable manually, but this seems hacky.
d

daniel

06/02/2023, 6:08 PM
Hi Josh - there are some recommendations here for how to do this: https://github.com/dagster-io/dagster/discussions/12183
although I think the approach there is more or less the solution you described
j

Josh Kutsko

06/02/2023, 6:09 PM
Ah, awesome, that looks like a less hacky solution though, doing it at the entry point is cleaner than us re-creating the file for every usage.
looks good to me, thanks
g

geoHeil

06/02/2023, 8:24 PM
have you looked into Workload identity?
plus1 2
j

Josh Kutsko

06/02/2023, 8:24 PM
Oh I haven't, will do
o

Oren Lederman

06/05/2023, 4:10 PM
+1 for workload identify. It’s a bit confusing to setup the first time - you need to create a GCP Service Account, a K8s Service Account and kinda point them to each other. But after you set it up, all you need to do is to tell your Dagster pods to use this K8s Service Account.
j

Josh Kutsko

06/05/2023, 4:11 PM
gotcha, ok. I checked on the github issue originally linked and since we’re using dagster cloud, it implies the workload identity solution won’t work. Is this still the case?
d

daniel

06/05/2023, 4:12 PM
That won't work in a Serverless deployment where we manage the execution plane, but would work in a Hybrid deployment
j

Josh Kutsko

06/05/2023, 4:12 PM
ah ok makes sense
5 Views