https://dagster.io/ logo
#announcements
Title
# announcements
d

David

03/11/2021, 1:09 PM
Hello, I have a question about the helm chart, where can I define the resource limits and requests used by jobs
💯 1
j

johann

03/11/2021, 1:24 PM
Hi @David: K8s allows you to set resource quotas and default resource requests for a namespace using ResourceQuotas and LimitRangers: https://kubernetes.io/docs/concepts/policy/resource-quotas/ You can then override the defaults for each part of your dagster deployment. Resources for for dagit/daemon/userDeployments can be specified in the helm chart. Resources for jobs can be specified on tags for specific solids https://docs.dagster.io/deploying/kubernetes#custom-kubernetes-configuration
If you’ll be using the same resource request tags across multiple solids, a recommended pattern is that you use a solid factory function that applies the tags
d

David

03/11/2021, 1:47 PM
thanks johann 👍
uhm still seems not to work
job that is executed still has default limit and requests of 1 cpu and 1 Gi memory
also tried to tag the pipeline with from dagster_k8s.job import K8S_RESOURCE_REQUIREMENTS_KEY
j

johann

03/11/2021, 2:00 PM
Are you using celery-k8s or just dagster-k8s?
d

David

03/11/2021, 2:00 PM
runLauncher: # Type can be one of [K8sRunLauncher, CeleryK8sRunLauncher, CustomRunLauncher] type: K8sRunLauncher
in the helm chart
j

johann

03/11/2021, 2:00 PM
Ah I was going to recommend tagging the pipeline, that didn’t work?
d

David

03/11/2021, 2:01 PM
nope
I use the following solid decorator
Copy code
@solid(
    tags={
        'dagster-k8s/config': {
            'container_config': {
                'resources': {
                        'requests': { 'cpu': '1', 'memory': '1Gi' },
                        'limits': { 'cpu': '2', 'memory': '8Gi' },
                }
            },
        },
    },
    required_resource_keys={'elasticsearch'},
    output_defs=[DynamicOutputDefinition()]
)
j

johann

03/11/2021, 2:03 PM
That looks correct,
kubectl describe job
doesn’t show any of the added config?
And to be clear, you placed the same tag on the @pipeline?
d

David

03/11/2021, 2:06 PM
no only in the solid
have to put them on both?
pipeline is defined as
Copy code
@pipeline(
    mode_defs=[
        ModeDefinition(
            resource_defs={
                'elasticsearch': es_resource,
                'elasticsearch_async': es_async_resource,
                'io_manager': fs_io_manager,
                'loop': loop_resource,
                # 'gcs': gcs_resource,
            },
            executor_defs=default_executors + [celery_executor],
        )
    ]
)
def aggregation_pipeline():
j

johann

03/11/2021, 2:07 PM
There’s an issue with the docs- for the
K8sRunLauncher
, we only create one k8s job for the entire run, so the tag should be on the pipeline. Sorry about the confusion
For the
CeleryK8sRunLauncher
, we create a job per solid so you can vary resources at the solid level
Created a tracking issue to fix the docs 🙂 https://github.com/dagster-io/dagster/issues/3824