Hi all 👋, I'm very new to Dagster and I'm working on deploying dagster along with a user code deployment to a local k8s cluster (using Helm). I'm using

K8sRunLauncher

along with the

k8s_job_executer

with a very simple 2 step pipeline. I'm able to get my pipeline to start, but I'm using GCS as my io_manager. Once my first step starts, I see this error message:

Initialization of resources [gcs, io_manager] failed.
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started>

My question is, what's the proper way to set this env variable? My thought is to take the following steps: 1. create a k8s secret that has the contents of my private key file. 2. define a k8s volume with the secret 3. mount the secret volume to the

k8s_job_executor

job's container 4. set

GOOGLE_APPLICATION_CREDENTIALS

to point to key file in volume mount I'm not exactly sure how to accomplish step 3 though. Does anyone know if this is the proper way to use GCS for intermediate storage? Is there a better way? Thanks in advance!

Hi @Ginger Balmat, unfortunately the volume_mounts config for k8s currently isn’t exposed in the k8s_job_executor. It’s a small change- could have it out in the release this thursday.

Thanks so much for the clarification @johann! I think what I'll do is use Minio locally and then rely on the service account for my GKE deployment.