https://dagster.io/ logo
Title
s

Sarah Gruskin

06/10/2021, 9:49 PM
Hello, I'm trying to setup a pipeline that is partitioned and has a schedule. I ended up making this stackoverflow question. Was wondering if anyone would be able to help. https://stackoverflow.com/questions/67928569/how-to-create-partitions-with-a-schedule-in-dagster
r

rex

06/10/2021, 9:59 PM
hey sarah - as a side note, it’s a little bit more ergonomic to define a schedule using one of our schedule based decorators: https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules#partition-based-schedules, since you’re defining a partition set based on dates
o

owen

06/10/2021, 10:09 PM
^ agree with what rex said, but what's going on with your specific code has a pretty simple fix. your get_date_partitions function is returning a list with a single
Partition
that contains a list of all the dates. It should probably be written something like this instead:
def get_date_partitions():
    return [
        Partition(datetime.strftime(TODAY - timedelta(days=x), DATE_FORMAT))
        for x in range(get_number_of_days().days)
    ]
This will return a list of partitions, representing a single day each. However, the schedule documentation that rex linked is really the easiest way to implement your desired behavior, so I recommend checking that out.
s

Sarah Gruskin

06/10/2021, 10:10 PM
That did the trick! Thanks so much! SO SIMPLE!
😛artydagster: 1