Hi Is there any way to combine preset and run config? I want to build pex zip of my pipeline with preset config yamls inside. And then do spark submit of this zip, but would like to override some values defined in that yamls. If i use pipeline execute cli i am getting error that preset and config cannot be specified at the same time. Thank you.

You can copy the preset to a dict with

config = prod_preset.run_config.copy()

then modify it or add keys as needed and use "config" as your new run config.

It seems that execute pipeline api requires preset to be passed and it should have the same run config

did you try to pass the mode and then the modified run config?

This api requires to pass preset It should be one of presets defined for pipeline If i pass mothing it throws exception

This is how I call it in a bunch of my tests

Do you have presets specified in my_solid?

yes

Yes it works Thanks a lot!