Doing a trial/PoC with dagster coming from an Airf...
# ask-community
Doing a trial/PoC with dagster coming from an Airflow background. Is there a way to know that all tasks in a failed run have ever been completed successfully? In Airflow there's the 'green' or 'red' dot that once all tasks have been retried successfully indicates it no longer needs attention. When I run a task that autofails 1 of 10 tasks, and then rerun that single task to succeed. It's now a separate Run that shows success. How do I mentally know that this pipeline is in a fully complete/healthy state?
You can take a look at the
dagit page (click “status” in the top right corner) to get a view of all the runs that have come from a job. If the rightmost pez is green, your job is in a healthy state.
Thanks! Let me check my test job and see what that looks like
If you’re looking for a view similar to airflow (e.g. if you’re partitioned by date), you could check out the partitions view for your job. We have docs here to make your job aware of these partitions.
So the instance overview doesn't appear to be 'correct' in terms of knowing if a previous run has been executed in full. For example in a test job.
Copy code
spawns 10 tasks, task 1 and 2 are set to always fail
Status/instance overview shows rightmost square as red. If I change the code so task 2 will pass. And rerun task 2. /instance for the job shows a successful run and the rightmost task is green. But nothing indicates that Task 1 hasn't be fixed
Thanks Rex, I'll check that out. I just started trying Dagster today so that's a bit beyond me. Sort of doing 'hello-world' style dags and dynamic outputs. Trying to understand how observability works for independent failed tasks in an dynamic job.