Deveshi
04/01/2021, 3:10 PMalex
04/01/2021, 3:14 PMrun_config
from the sensor when you return the RunRequest
this example should be sort of relevant
https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules#using-a-preset-in-a-schedule-definitionDeveshi
04/01/2021, 3:23 PMalex
04/01/2021, 4:16 PMPresetDefinition
right? The idea here is to be able to reference that PresetDefinition
instance from your run request. If you are defining the object directly on your pipeline you’ll have to move it so you can reference it from both placesDeveshi
04/01/2021, 4:16 PMdefault = ModeDefinition(
name="default",
resource_defs={"s3": s3_resource_for_profile, "redshift": redshift_resource})
#main pipeline
@pipeline(mode_defs=[default],
preset_defs =[
PresetDefinition.from_files(
name="default",
mode="default",
config_files=["resources.yaml"]
)
]
)
def example_pipeline():
alex
04/01/2021, 4:17 PMModeDefinition
you’ll have to move PresetDefinition
so you can reference it and call run_config
on the instanceDeveshi
04/01/2021, 4:21 PM