Zach06/20/2022, 6:31 PM
sandy06/20/2022, 10:26 PM
are you saying that there's no Execution Time graph for
@graph def my_graph(): upstream.map(downstream)
? the reason we don't show a graph for
is that it's usually multiple steps, and those steps might not be executing contiguously. e.g. the last step might complete long after the first step starts because it had to wait in a queue for resources on the cluster to open up. do you have thoughts on the ideal behavior?
Zach06/21/2022, 5:35 PM
op that I noticed had no Execution Time graph. your reasoning makes sense, I didn't really think about where you'd put the op executions on the X axis. I was hoping I'd be able to get a view of execution time from op start to op finish across all executions of the op, but maybe that'd be more appropriate for a different view or an external monitoring service. I guess more generally it'd be great to have more ways to graphically view job / op execution time and status over historical timelines... things like number of job runs in each status for a given job, number of a particular op running across all job runs for a given job, number of jobs. how many runs where there for a given job over the last X days, how many failed? what was the average completion time of successful job runs for a given job over the last X days? in 1 hour windows what was the average number of running ops for a given job across all runs? in the same window how many were in other states?
sandy06/21/2022, 9:30 PM
josh06/21/2022, 9:58 PM