Ismael Rodrigues
07/26/2023, 5:21 PMsandy
07/28/2023, 11:32 PMIsmael Rodrigues
07/29/2023, 1:07 PM@op
def my_op(context, config: MyClassConfig):
pass
This op will require from you to define in somewhere:
{
"ops": {
"my_op": {
"config": {
"date": '2023-01-01'
}
}
}
Something like this, and the point is, that if you have this config in your OP, you can't define the configuration to limit concurrency of OPs, because it'll require that you also configure this other thing that is supposed to be configured by schedule, sensor or partition definitionsandy
07/30/2023, 3:15 PM@job(executor_def=multiprocess_executor.configured({"max_concurrent": 4, ...}))
def tag_concurrency_job():
...