Stephen Bailey
08/08/2022, 1:59 PMrun_config
dict when creating the job, which produces a default configuration in the jobs pane. This is nice because it provides a set of runtime defaults for end users to view.
With assets, it seems that there is no way to provide a set of default config for materializations. So the options are to either:
1. Create a job that materializes the asset and provide config there. But, this doesn't allow users to click "Materialize" in Dagit without passing in additional config.
2. Pass in default values to the config_schema
, which allows the "Materialize" button to work, but does not show config in the UI.
Do I have that right? My use case right now is that I have an asset that is produced via an AWS Sagemaker Training Job, and I want to allow data scientists to both see and override the training instance size and other runtime parameters for individual training runs.Vinnie
08/08/2022, 2:16 PMwith_resources
function, but this doesn’t seem to show up on the interface. It could just be an odd interaction in my case since I’m defining the assets with the AssetDefinition.from_graph()
function. Also interested to know if there’s a better way.
@repository
def repo():
return [
*with_resources(
definitions=[my_cool_asset],
resource_defs={
"source": my_cool_source,
},
resource_config_by_key={
"source": {"config": {"my_cool_parameter": "my_cool_value"}},
},
)
]
sandy
08/08/2022, 3:21 PMStephen Bailey
08/08/2022, 3:22 PMVinnie
08/08/2022, 3:23 PMsandy
08/08/2022, 3:37 PM