Rasmus Bonnevie
04/07/2023, 7:09 AMcomplex_config_x
, and a function config_to_asset(cfg)
, how do I turn this into 10 assets in the most dagsteronic way? Factories? Configured assets? Graph assets with configured ops? Partitions?Nicolas Parot Alvarez
04/07/2023, 10:02 AMDavid Merritt
04/07/2023, 4:06 PMRasmus Bonnevie
04/07/2023, 4:09 PMsandy
04/07/2023, 6:54 PMLeo Qin
04/07/2023, 8:57 PMcompute_kind
label to denote what kind of template it's following. It's a nice visible label and it doesn't force users to stick to a certain key prefix or asset group.Rasmus Bonnevie
04/10/2023, 9:00 AMconfig = PydanticModel()
def chain_factory(config, last_chain_asset):
@asset(
ins={"last_chain_link": AssetIn(key=last_chain_asset.key)},
metadata={"config": MetadataValue.json(config.dict())},
)
def chain_node_asset(last_chain_link):
# config passed as closure
return asset_from_config(config, last_chain_link)
next_asset = chain_factory(config, first_asset)
but that means that we completely bypass all notions of dagster configuration and only keep an informal record in the metadata. Could this be done otherwise?