Doing a trial/PoC with dagster coming from an Airf...
# ask-community
j
Doing a trial/PoC with dagster coming from an Airflow background. Is there a way to know that all tasks in a failed run have ever been completed successfully? In Airflow there's the 'green' or 'red' dot that once all tasks have been retried successfully indicates it no longer needs attention. When I run a task that autofails 1 of 10 tasks, and then rerun that single task to succeed. It's now a separate Run that shows success. How do I mentally know that this pipeline is in a fully complete/healthy state?
r
You can take a look at the
/instance/overview
dagit page (click “status” in the top right corner) to get a view of all the runs that have come from a job. If the rightmost pez is green, your job is in a healthy state.
j
Thanks! Let me check my test job and see what that looks like
r
If you’re looking for a view similar to airflow (e.g. if you’re partitioned by date), you could check out the partitions view for your job. We have docs here https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#partitioned-jobs to make your job aware of these partitions.
j
So the instance overview doesn't appear to be 'correct' in terms of knowing if a previous run has been executed in full. For example in a test job.
Copy code
spawns 10 tasks, task 1 and 2 are set to always fail
Status/instance overview shows rightmost square as red. If I change the code so task 2 will pass. And rerun task 2. /instance for the job shows a successful run and the rightmost task is green. But nothing indicates that Task 1 hasn't be fixed
Thanks Rex, I'll check that out. I just started trying Dagster today so that's a bit beyond me. Sort of doing 'hello-world' style dags and dynamic outputs. Trying to understand how observability works for independent failed tasks in an dynamic job.