Hi team, we are trying to get our Dagster jobs run...
# deployment-kubernetes
z
Hi team, we are trying to get our Dagster jobs running under Kubernetes namespaces using
k8sRunLauncher
, but we couldn't get it fully work. After the configuration, we got the ops running under the namespace with the pod name as
dasgter-step-<uuid>
. However, the job pod is still running under the default namespace with the pod's name as
dagster-run-<uuid>
. We have configured the execution config for the job to pass the
job_namespace
attribute, and its value should be dynamic depending on which tenant we are using. Do you have any ideas if we have missed any other configuration? Below is our execution config:
Copy code
execution:
  config:
    env_secrets:
    - tenantX-secret
    job_namespace: tenantX
ops:
	op_1: ...
	op_2: ...
d
Hi zack - the executor only controls which namespace the steps run in, not the namespace that the Dagster run happens in (confusingly, the 'job_namespace' field there is referring to a kubernetes job for the step, not a Dagster job. By default, I think the run pod will happen in the same namespace as your user code deployment (assuming you're using the helm chart). Is that different than what you want?
I don't think we actually currently have a way to vary the namespace that's used for the dagster-run pod using tags or code
z
Hi @daniel, we actually have user code deployment under different namespaces, but we are not using the helm chart to deploy the user code since we have different user codes under different namespaces for each tenant. I have looked into the code which is showing we can configure the
UserDefinedDagsterK8sConfig
by passing the tag including namespace under
the job_metadata
. I was expecting to see the namespace passing on the tag will overwrite the default namespace. However, when we launched the job, it shows unmatched error: HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the namespace of the provided object does not match the namespace sent on the request","reason":"BadRequest","code":400}. I guess the job namespace might be coming from the helm charts setting where we configure the Launcher. We are still looking for a way to configure the namespace for the dagster-run from the tag. Do you think there will be a release for Dagster having this feature?
d
Would it be possible to file a feature request issue on GitHub for configuring the namespace via tags?
z
Sure Daniel, I will do that. Thanks for your feedback!
k
Hi Daniel , do we have any update on this ? Is there a way now to run the dagster-run pods under a namespace rather than the default one ? I am Zack's colleague and needed to implement it for my Project.