Hi team, we just upgraded to 0.12.3 from 0.11.x. W...
# ask-community
Hi team, we just upgraded to 0.12.3 from 0.11.x. We noticed that our partition-based daily schedule kicked off pipeline runs over the the weekend even though the partition selector we use generated partitions that have already been run, as evidenced by the partition and run_keys shown below - this wasn't happening before. Although we have
cron_schedule="0 3 * * *"
in our schedule, the weekend runs were previously inhibited, I presume due to the run_key having already been used? We can (and should), of course, change the cron_schedule to add MON-FRI, but did we just fluke the desired behaviour before 0.12.x?
@daniel - is this familiar to you?
Is it possible to see the partition selector? I'm a little rusty on how exactly run keys work within schedules
Ok, I just checked - I think setting a run key within a schedule only disambiguates runs within a single tick (so if your schedule did two runs each day, run keys would let you be sure that you only ever got those two runs on that day no matter what). So yeah, the mystery here is why these were ever skipping the weekend..
it's possible we used to be checking for duplicate partitions more explicitly within the scheduler before, that seems more likely than run keys to me. At the risk of passing the buck again @prha may have context here too