Testing multiple models inside a workflow Hello, i wonder if dagster fit my needs for this use case. I have a workflow with multiple models A->B->C etc. Of course we have to test multiple model B and C (for exemple) and the final results is the most important. Having the best B model does not lead at any case the best results since it's a coupling of everymodel. Do you think dagster is made for this ? If yes, can I have a running workflow and update it to replace / add a branch to it ? I have a new B model which update everything downstream. Those question can be solved by let's say Jenkins but I wonder which "orchestrator" will be the most efficient

Hey @Thomas - when you say "model", are you talking about ML models? When you say "test", are you talking about testing like in unit testing, or trying different versions out in a data science setting to determine which gives better performance? In either case, Dagster could be a good fit. If this works in Jenkins, it would likely work in Dagster as well.

Thanks for your questions Models will be mainly ML models and test will be for performance of the global process. Do you have in exemple or solething similarité to it ? What I think I will look at is to trigger action when one step is changing (a model is updated via a webhook for ex). Add a Split in the middle of the workflow to test in parallel another branch with the same input against the same outputs

I don't think we have examples for this directly, but the guide we have on constructing pipelines out of solids might be helpful - https://docs.dagster.io/tutorial/basics_solids

I started with this. But didnt see how to restart only what I need or trigger event. I will look deeper :)