alex
07/08/2019, 3:23 PMDo you have any plans to make, say, an aggregated report of failed expectations?We are currently working on rendering the dagster events in a structured and much richer way in the log viewer. This combined with better filtering options should approximate this. Good feedback and something we'll keep in mind.
What is you recommendation for this situation?Great question, expectations are something we are super excited about and are actively iterating on quite a bit. I think if you always want these checks to cause the computation to cease - then raising
Failure
might be the right choice. Failure
can report the same metadata that ExpectationResult
can. If you want to be able to vary whether the computations continue or fail - then dagster having a way to toggle whether ExpectationResult
failures cause the step to fail makes a lot of sense (we would need to add this). In the short term I advise writing a little helper function that can do either ie yield expectation_check(check_result, metadata)
so you only have one place to change as you continue experimenting.jack
07/08/2019, 5:00 PM