Hi team, just want to confirm that if I use

op_tags

like this and select to materialize the asset, the asset job will use this config, right? I’m currently not seeing this, not sure what is wrong

@multi_asset(
     ...
    op_tags={
        "dagster-k8s/config": {
            "container_config": {
                "resources": {
                    "requests": {"cpu": "5000m", "memory": "64Gi"}
                },
            },
        }
    }
)
def test_asset(context: OpExecutionContext) -> pd.DataFrame:
    pass

Hi Sean, this tag would need to go on the job that's being launched, yeah. Putting it on an individual asset (even if its multi-asset) isn't going to cause it to the affect the job that's being run

I usually select the asset and click materialize, is there a way to let Dagster know that it needs the tag? Or I have to convert this multi asset to a job in code in order to apply the tag?

I think for this you would need to call define_asset_job, yeah

In this doc it says if I use

k8s_job_executor

, I can also use the

dagster-k8s/config

tag on a Dagster op to control the Kubernetes configuration for that specific op. Does this apply to asset? https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#instance-level-kubernetes-configuration

yeah, in that case, applying op_tags would be applied to that op

So my question is, I setup my repo as

*@repository*(default_executor_def=k8s_job_executor)

, is what I described above mean that tag is not applied? or applied and I can’t see it

I'm not sure that's what you want for materializing a single asset though - in that case, it would spin up a k8s pod for the run (that doesn't do very much) and then a k8s pod just for running the single asset

Yes, I want a pod with more memory to materialize that asset since that asset is big

Yeah, I think this is exposing a thing that should be easier in the current system.... some kind of way to apply tags to jobs that materialize just this one asset could help potentially?

Could you explain more about the second solution? I don’t fully understand. Part of the reason I don’t want to use an asset job is that I want to be able to select a range of partition and materialize those, which I think job is not capable of.

The k8s_job_executor solution you mean?

Yes, is there anything I need to do differently for the tag to apply?

No, I think op_tags would work with the k8s_job_executor. You wouldn't see it in the tags for the run because that doesn't include op tags, but i would expect the pods for each individual step/asset to have the resource limits applied

Thanks! Good to know that it is applied