I am not sure this is the right place to ask Please remove i

Question

I am not sure this is the right place to ask Please remove if this doesn t belong here I would like to visualize the task duration And I have no knowledge of Air Flow and I am thinking of just reading the log file and plotting it on Dash or something similar Could anyone point me in the right direction or approach Currently I have no clue how to save from Dagit Thanks

abhi · Answer

Hi Thein So when you say task duration do you mean the duration of the pipeline execution run or the duration of a particular solid execution

abhi · Answer

Regardless `execute pipeline` returns a pipeline execution result object This object contains event metadata about your pipeline run Upon inspection of those step events it should become fairly easy to grab whatever metadata you want and write it to a table or pipe it out to a file somewhere but if you want more you can check out the DagsterEvent object to learn more

abhi · Answer

I would also love to learn more about your dash boarding use case What metrics are you looking to track

Thein · Answer

Hi Abhi thanks for the reply This is really helpful As of now I just want to monitor which ML models run faster and compare between each run It is very fundamental at this point And another use case is to monitor how many each model run every day

abhi · Answer

That makes total sense This should be sufficient for answering those flavors of questions If not let us know and we can go deeper