Henry
03/09/2021, 11:37 PM@repository
def tmp_pipeline_repository():
tmp_pipeline = pipeline_factory()
return [tmp_pipeline, tmp_pipeline_schedule]
where pipeline_factory
returns a pipeline whose functionality changes based on some external data which changes asynchronously from the pipeline, will the pipeline be reconstructed before each scheduled run? If not, is there a way to do this that conforms to dagster
design patterns? Happy to clarify the details further if this is too vaguedaniel
03/10/2021, 12:00 AMHenry
03/10/2021, 12:01 AM