@Scout Is there a way to add a dnsConfig to my dagster helm deployment. I have a main dagster deployment and then user deployments. The thing is that i want to use a different nameserver as the default one can't resolve DNS.

The documentation does not explicitly mention how to add a

dnsConfig

to a Dagster Helm deployment. However, it does provide general guidance on customizing your Kubernetes deployment with Dagster, including specifying custom Kubernetes configurations for instance-level, per-job, and even individual steps within a run. To customize the Kubernetes deployment, you would typically modify the Helm chart values, which can include various Kubernetes-specific configurations. The

dnsConfig

is a standard Kubernetes pod specification, so you would need to include it in the appropriate section of your Helm chart values. While the documentation does not provide a specific example for

dnsConfig

, you can usually add such configurations in the

podSpecConfig

section of your Helm values file. Here is a general example of how you might include a `dnsConfig`:

yaml runLauncher:   type: K8sRunLauncher   config:     kubeconfigFile: ...     loadInclusterConfig: ...     # ... other configurations ...     runK8sConfig:       podSpecConfig:         dnsPolicy: "None" # Example policy         dnsConfig:           nameservers:             - "your-custom-nameserver"           searches:             - "your-custom-search"           options:             - name: "ndots"               value: "2"             # ... other dnsConfig options ...

Please note that this is a general Kubernetes configuration and not specific to Dagster. You would need to adjust the Helm chart values according to your specific requirements and the structure of the Dagster Helm chart you are using. If you need further assistance with this, I would recommend checking the Helm chart documentation for Dagster or reaching out to the community for more specific guidance, as the customization of the Helm deployment can vary based on the version of the chart and your particular Kubernetes environment.

@Scout I just want to add more nameservers to the overall deployment

To add more nameservers to your overall Dagster Helm deployment, you would typically modify the Helm chart values to include a

dnsConfig

with the additional nameservers. This configuration would be part of the

podSpecConfig

for the relevant components of your deployment, such as the Dagster webserver, daemon, and user code deployments. Here's an example of how you might specify additional nameservers in your Helm values file:

yaml # Example Helm values configuration webserver:   deployment:     podTemplateSpec:       spec:         dnsConfig:           nameservers:             - "1.1.1.1"             - "8.8.8.8"             - "your-custom-nameserver"  daemon:   deployment:     podTemplateSpec:       spec:         dnsConfig:           nameservers:             - "1.1.1.1"             - "8.8.8.8"             - "your-custom-nameserver"  userDeployments:   enabled: true   deployments:     - name: "user-code-deployment-1"       podTemplateSpec:         spec:           dnsConfig:             nameservers:               - "1.1.1.1"               - "8.8.8.8"               - "your-custom-nameserver"     # Add additional user code deployments as needed

In this example,

1.1.1.1

and

8.8.8.8

are placeholders for public DNS servers, and

your-custom-nameserver

should be replaced with the IP address of the nameserver you want to use. You would add this configuration under each component that you want to customize. Please ensure that you are using the correct structure for your specific version of the Dagster Helm chart, as the structure and keys may vary. After updating the values file, you would apply the changes by upgrading your Helm release with the updated values:

shell helm upgrade <release-name> dagster/dagster -f values.yaml

Replace

<release-name>

with the name of your Dagster Helm release, and

values.yaml

with the path to your modified values file. Remember to review the Helm chart documentation and Kubernetes documentation on

dnsConfig

to ensure that the configuration is correct and supported by your Kubernetes cluster version.

@Scout It looks like this now:

runK8sConfig: 
        podSpecConfig:
          dnsConfig:
            nameservers:
              - "1.1.1.1"
              - "8.8.8.8"
          nodeSelector: {}
        jobSpecConfig: # raw config for the kubernetes job's spec
          ttlSecondsAfterFinished: 1

Based on the snippet you've provided, it appears that you've correctly added a

dnsConfig

section under

podSpecConfig

within the

runK8sConfig

of your Helm values file. This configuration should apply the specified DNS settings to the pods launched by the Dagster

K8sRunLauncher

. Here's a breakdown of your configuration: - `runK8sConfig`: This is the configuration that will be applied to the Kubernetes objects created by the

K8sRunLauncher

. - `podSpecConfig`: This section allows you to specify configurations that will be applied to the

PodSpec

of the pods launched by Dagster. - `dnsConfig`: This is a standard Kubernetes configuration that specifies DNS settings for the pods. - `nameservers`: Under this key, you list the DNS servers that you want the pods to use. In your case, you've specified

1.1.1.1

and

8.8.8.8

. - `nodeSelector`: This is an empty dictionary in your configuration, which means no specific node selection rules are applied. You can specify node labels here if you want to constrain your pods to run on specific nodes. - `jobSpecConfig`: This section allows you to specify configurations for the

JobSpec

of the Kubernetes Jobs created by Dagster. - `ttlSecondsAfterFinished`: This setting specifies the number of seconds to wait before cleaning up the completed Jobs. In your case, it's set to

1

, which means Jobs will be cleaned up one second after completion. If you want to apply this DNS configuration to all components of your Dagster deployment, you would need to ensure that this

dnsConfig

is included under the

podSpecConfig

for each component (e.g., webserver, daemon, user code deployments) in your Helm values file. After updating your Helm values file, you would apply the changes by upgrading your Helm release with the updated values:

shell helm upgrade <release-name> dagster/dagster -f values.yaml

Replace

<release-name>

with the name of your Dagster Helm release, and

values.yaml

with the path to your modified values file. Please make sure to review the Helm chart documentation for Dagster and the Kubernetes documentation on

dnsConfig

to ensure that the configuration is correct and supported by your Kubernetes cluster version. If you encounter any issues or have further questions, we're here to help!