How to configure kubernetes helm chart, so that runs will get deleted when completed

To configure the Kubernetes Helm chart so that runs will get deleted when completed, you can utilize the TTL (Time To Live) feature provided by Kubernetes. This feature automatically deletes completed Jobs (and their associated Pods) after a certain period. Here's how you can set it up according to the documentation: 1. Enable TTL Controller for Kubernetes Cluster: If you are using a Kubernetes version that supports the TTL Controller, ensure that it is enabled. For Kubernetes versions prior to 1.21, you may need to enable the feature gate

TTLAfterFinished

for your cluster. 2. Modify Helm Chart Values: You will need to modify the values in your Helm chart to include the

ttl_seconds_after_finished

configuration. This can be done by setting the TTL value in the job tags within your Dagster job definition. Here is an example from the documentation:

python    @job(        tags = {            'dagster-k8s/config': {                'job_spec_config': {                    'ttl_seconds_after_finished': 7200  # Sets the TTL to 2 hours                }            }        }    )    def my_job():        # Define your job's logic here

In this example,

7200

is the TTL value in seconds, which means that the completed jobs will be deleted after 2 hours. You can adjust this value to your preferred TTL duration. 3. Apply the Helm Chart: After modifying the values, apply the changes by upgrading your Helm release with the updated values file:

shell    helm upgrade --install user-code dagster/dagster-user-deployments -f /path/to/values.yaml

By following these steps, you can configure your Dagster Helm deployment to automatically delete completed runs after a specified TTL. This helps in managing resources and keeping your Kubernetes cluster clean from completed job artifacts.