Hi team! Is there a way for us to modify asset level Kubernetes configuration? Let’s say I have an asset that is auto-materialising eagerly. It requires an abnormally high amount of memory to operate on a dataset. Is there a way to apply custom Kubernetes configuration to request for higher memory whenever it auto-materialises? So far from the doc, it appears that I need to materialise this asset with a job and use

k8s_job_executor

to run each step in its own pod with custom Kubernetes configuration.

Hi Bolin - if you're using the

k8s_job_executor

you can provide

op_tags

when you're defining your assets

Thanks Sandy! After providing

ops_tags

when defining the asset, the asset must be run by a job in order to those custom configurations to be applied right? If the asset were to auto-materialise, the supplied

op_tags

is doing nothing.

For this to work you would need to specify the 'executor' field on your Definitions object in order to make it the default executor used by the jobs that are auto-materialized, yeah. Which would also have the effect of making each of your assets that are auto-materialized use that executor - we don't yet have support for mixing and matching the executors that are used by auto-materialized assets (i.e. run most assets in a subprocess, but just one in a k8s job) - but it's on our radar as something we'd like to support in the future

Thanks a lot Daniel! This is very helpful!