Curious, in dagjt, is there an option where for the selected pipeline, only the schedules related to that pipeline are listed? Currently, it lists all schedules regardless of chosen pipeline. I guess that might be tough since currently there isnt a way to unselect a pipeline, and thus have the option to also see all schedules.
01/24/2021, 4:11 PM
There is currently not. We are considering adding more lightweight tagging and filtering where users could set tags alongside definitions and then filter them in the UI
Curious, what is your schedule-to-pipeline ratio?
01/24/2021, 4:32 PM
Generally 1-1, but I do have a pipeline where it has 3, only because for testing, Im trying to see how small of a scheduled interval I can achieve. Which is nice about dagster and a nice unexpected surprise for me that i can change schedule mid-execution. But yeah, for production, I would think the ratio would be 1-to-1. So unless I have several pipelines, there isnt a need for a filtering. Will look into the tagging feature that you mentioned.
01/24/2021, 4:56 PM
Oh interesting. What do you mean by “mid-execution” exactly?
01/24/2021, 5:23 PM
I had a pipeline running every 1 minute, while it was executing, I switched to a 5 minute interval and it did after it finished the one min schedule, so maybe misleading, so didnt switch before the one minute schedule completed, but waited until it did. So what I meant is during the middle of the dagster instance runtime, not middle of that one minute schedule. Yeah i can see it being a bad thing if it quits that one min schedule without finishing and then moving on to the 5 min schedule. I apologize if that was the impression made. But it was great that I can switch to a diff schedule.