Chaitya
04/26/2023, 1:22 AMmaterialize
but can't seem to get it working with the new pythonic API. Details in thread.class MyIOManagerResource(ConfigurableResourceFactory[MyIOManager]):
base_path: str
def create_resource(self, context: InitResourceContext) -> MyIOManager:
return create_my_io_manager(self.base_path)
class MyResource(ConfigurableResource["MyResource"]):
some: str
fields: str
In my call to materialize previously, I would construct a config dictionary and call materialize like this:
run_config = {
"resources": {
"io_manager": {
"config": {
"base_path": "/tmp"
}
},
"my_resource": {
"config": {
"some": "value1",
"fields": "value2",
}
}
}
}
_ = materialize([my_asset], run_config=run_config, resources={ "io_manager": create_my_io_manager, "my_resource": MyResource.configure_at_launch()})
This worked as I expected.
Now trying to switch to the new pythonic configuration:
_ = materialize([my_asset], run_config=RunConfig(resources={"io_manager": MyIOManagerResource(base_path="/tmp"), "my_resource": MyResource(some="value1", fields="value2")}), resources={ "io_manager": create_my_io_manager, "my_resource": MyResource.configure_at_launch()})
This fails with:
Error 1: Received unexpected config entry "my_resource" at path root:resources. Expected: "{ io_manager?: { config?: Any } }".
If I remove the my_resource
key in the RunConfig constructor:
_ = materialize([my_asset], run_config=RunConfig(resources={"io_manager": MyIOManagerResource(base_path="/tmp")}), resources={ "io_manager": create_my_io_manager, "my_resource": MyResource.configure_at_launch()})
this runs successfully. But unfortunately, I cannot run this without my_resource
. Is this a bug? How can i pass multiple resources to the materialize call? I'm not sure I need the MyResource.configure_at_launch()
anymore, but i'm not really sure if thats causing my issue.chris
04/26/2023, 5:39 PM