Zach
06/20/2022, 6:31 PMsandy
06/20/2022, 10:26 PM@graph
def my_graph():
upstream.map(downstream)
are you saying that there's no Execution Time graph for downstream
? the reason we don't show a graph for downstream
is that it's usually multiple steps, and those steps might not be executing contiguously. e.g. the last step might complete long after the first step starts because it had to wait in a queue for resources on the cluster to open up. do you have thoughts on the ideal behavior?Zach
06/21/2022, 5:35 PMdownstream
op that I noticed had no Execution Time graph. your reasoning makes sense, I didn't really think about where you'd put the op executions on the X axis. I was hoping I'd be able to get a view of execution time from op start to op finish across all executions of the op, but maybe that'd be more appropriate for a different view or an external monitoring service.
I guess more generally it'd be great to have more ways to graphically view job / op execution time and status over historical timelines... things like number of job runs in each status for a given job, number of a particular op running across all job runs for a given job, number of jobs. how many runs where there for a given job over the last X days, how many failed? what was the average completion time of successful job runs for a given job over the last X days? in 1 hour windows what was the average number of running ops for a given job across all runs? in the same window how many were in other states?sandy
06/21/2022, 9:30 PMjosh
06/21/2022, 9:58 PM