19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hi all :wave: I'm attempting to deepcopy a ModeDef...
# announcementse
esztermarton
03/19/2021, 7:08 PMHi all 👋 I'm attempting to deepcopy a ModeDefinition object. It's giving me an error. Am attaching the code snippet - would be great to understand - is it wrong to be doing this? I want to do it so that I can have a default mode defined, and allow pipelines to customise this (without mutating the original object).
esztermarton
03/19/2021, 7:09 PMa
alex
03/19/2021, 8:38 PMhm im not sure what exactly is going wrong there with the deepcopy - i am guessing something deep in the config schema system
alex
03/19/2021, 8:39 PMwhat are you trying to add to the mode copies? new resources?
e
esztermarton
03/19/2021, 8:39 PMexactly
esztermarton
03/19/2021, 8:40 PMand maybe in the future i'll want to remove resources too 🤔
a
alex
03/19/2021, 8:40 PMModeDefinitionnamedtuplelistdictse
esztermarton
03/19/2021, 8:40 PMi imagine the rest won't change very frequently
esztermarton
03/19/2021, 8:42 PMYeah, I did that as a workaround - I wrote util function - this look ok to you?
Copy code
def _copy_mode_definition(mode: ModeDefinition):
"""
Creates a copy of the mode definition.
"""
return ModeDefinition(
name=mode.name,
resource_defs=copy(mode.resource_defs),
executor_defs=copy(mode.executor_defs),
logger_defs=copy(mode.loggers),
intermediate_storage_defs=copy(mode.intermediate_storage_defs),
description=mode.description,
)a
alex
03/19/2021, 8:45 PMyep - nice i was just writing something similar
alex
03/19/2021, 8:46 PM Copy code
def adjust_resources(
mode: ModeDefinition,
new_resources: Dict[str, ResourceDefinition],
):
resource_defs = dict(mode.resource_defs)
for key, resource in new_resources.items():
resource_defs[key] = resource
# _replace is a tuple method that copies over all other properties
return mode._replace(resource_defs=resource_defs)e
esztermarton
03/19/2021, 8:47 PMBeautiful 👌 Thank you Alex
a
alex
03/19/2021, 8:47 PMthe other thing to consider is refactoring to late bind the mode
alex
03/19/2021, 8:48 PMso instead of working from a base mode that you make copies of - you work form a resource_defs dictionary that you bind into a mode only once you attach it to pipeline def
🤔 1
👍 1
alex
03/19/2021, 8:48 PMor something like that
e
esztermarton
03/19/2021, 8:51 PMThanks, I'll ponder on that too to see if that's nice in our setup. Just trying to keep it as simple as possible 🙂
2 Views