Not sure if understand it. But does dagster currently works with loops?
I work with time series forecasting and for some problems I need to iteratively predict future timesteps. Like on the pic. Can I make dagster display my features and still be able to have that loop?
Maybe is it possible to orchestrate pipelines: run features and model inside small pipeline and trigger it from big one?
06/18/2020, 2:24 PM
I dont think that would be supported in a pipeline as it would break the DAG
06/18/2020, 10:59 PM
Robert is correct - Dagster expects no cycles in the compute graph. You could, however, take the outputs of a pipeline run and use them as the inputs to a subsequent pipeline run