Basil V
03/25/2020, 11:02 PMexecute_pipeline
with step_keys_to_execute
defined in the RunConfig
however, I'm running into this error:
dagster.core.errors.DagsterStepOutputNotFoundError: When executing transform.compute discovered required outputs missing from previous step: [StepOutputHandle(step_key='load.copy_django_logs_to_redshift.compute', output_name='result')]
I thought maybe I could fix it by passing in rexecution_config
to RunConfig, but when I do I get this error:
TypeError: __new__() got an unexpected keyword argument 'rexecution_config'
Could someone give me pointers on how to run a specific pipeline step that depends on previous steps? Thanks so much really appreciate all the help.alex
03/25/2020, 11:04 PMexecute_solid
to be appropriate if you are trying to test a single solidBasil V
03/25/2020, 11:06 PMalex
03/25/2020, 11:11 PMpipeline.build_sub_pipeline([solid_name])
might do the trickBasil V
03/25/2020, 11:16 PMalex
03/25/2020, 11:17 PMexecute_pipeline(pipeline.build_sub_pipeline([solid_name]), ...)
- which assumes the mode you need is already on the full pipelineBasil V
03/25/2020, 11:17 PMBasil V
03/25/2020, 11:23 PMalex
03/25/2020, 11:24 PMstep_keys_to_execute
businessBasil V
03/25/2020, 11:24 PMbuild_sub_pipeline
seems error if I pass in the config for the entire pipeline, rather than passing in only the config for that solid. I think I can work around this but it would be nice if it just knew how to disregard the config for other solids w/out erroringalex
03/25/2020, 11:31 PMBasil V
03/25/2020, 11:33 PM