I am searching for something like the dagster cloud yaml app

Question

I am searching for something like the dagster cloud yaml approach <https docs dagster io dagster cloud deployment agents kubernetes configuration reference per location configuration> bot for OSS

daniel · Answer

Hey Grigorii it s not exactly the same but configuration on the user deployments in the helm chart is kind of like that in that configuration that you set there will be passed through to the runs as well <https docs dagster io deployment guides kubernetes deploying with helm step 6 configure your user deployment>

daniel · Answer

By default this configuration will also be included in the pods for any runs that are launched for the code location server You can disable this behavior for a code location server by setting `includeConfigInLaunchedRuns enabled` to `false` for that server

Grigorii Kushnir · Answer

< daniel> thank you for the answer I have noticed this but I am not using helm charts I m writing plain kube configs Can you tell me please how is this configuration applied for the runs actually

Grigorii Kushnir · Answer

What I m doing right now is that I just use a configMap with run launcher configuration that is mounted as dagster yaml in the dagit pod

daniel · Answer

It sets a DAGSTER CONTAINER CONTEXT environment variable on the user code deployment <https github com dagster io dagster blob master helm dagster charts dagster user deployments templates deployment user yaml L58 L60>

daniel · Answer

to a JSON object with this format <https github com dagster io dagster blob master helm dagster charts dagster user deployments templates helpers helpers tpl L97 L138>

Grigorii Kushnir · Answer

Wow So I can fit all my configuration into the `DAGSTER CONTAINER CONTEXT` env variable on the grpc server pod and this configuration will be applied for the runs with the grpc server code

daniel · Answer

that s right

Grigorii Kushnir · Answer

I will try that ASAP thank you very very much

Grigorii Kushnir · Answer

dancing

Grigorii Kushnir · Answer

< daniel> tell me please if there are any other options that force the dagster to use that `DAGSTER CLI API GRPC CONTAINER CONTEXT` variable For some reasons it is just ignored in my case

daniel · Answer

i m not aware of anything else that uses it we have tests in place that verify that it works

Grigorii Kushnir · Answer

For whoever will run into the same issues as me the `DAGSTER CLI API GRPC CONTAINER CONTEXT` env variable actually overrides the run launcher K8sRunLauncher configuration that you specify in `dagster yaml` If you remove the run launcher K8sRunLauncher configuration from the `dagster yaml` you will end up having DefaultLauncher for the code locations and the `DAGSTER CLI API GRPC CONTAINER CONTEXT` variable with the k8s configs ignored

daniel · Answer

yeah to clarify this is additional configuration for a particular code location it s not intended to replace the run launcher config