https://dagster.io/ logo
#announcements
Title
# announcements
k

Klaus Stadler

02/03/2021, 2:14 PM
is there a way to combine presets with e.g. the config generated from a schedule or partition, so you don't have to define secret envs in multiple places for the same pipeline?
d

daniel

02/03/2021, 2:28 PM
Hi Klaus- does the example here of bringing in a preset into a schedule function answer your question? https://github.com/dagster-io/dagster/discussions/2943
k

Klaus Stadler

02/03/2021, 2:34 PM
this might be it... but I don't understand how the date from the schedule is passed into the preset
d

daniel

02/03/2021, 3:06 PM
Presets don't take in inputs, so if that's a requirement that's probably not the right thing to use here. Is there a benefit to using a preset vs. having a shared helper function that each of the schedules call to generate config?
k

Klaus Stadler

02/03/2021, 4:06 PM
sorry, I probably expressed it wrong: In the scheduler and partitions I want to take the preset as the base configuration and then add the from date and to date dynamically. Is this possible with above method?
d

daniel

02/03/2021, 4:47 PM
Doing this is possible, yeah - the preset can supply a run config dict, and then in the schedule function you can get it and then modify it however you like.
d

dhume

03/22/2021, 3:10 PM
Just curious on this. I tried to use a dynamic date within a schedule
Copy code
demo_schedule = ScheduleDefinition(
    name="DemoSchedule",
    cron_schedule="*/2 * * * *",
    pipeline_name="demo_pipeline",
    run_config={
        "solids": {
            "second_power": {
                "inputs": {
                    "date": {
                        "value": datetime.datetime.now().strftime(
                            "%m/%d/%Y, %H:%M:%S"
                        )
                    },
                }
            },
        },
    },
    mode="dev",
)
And it wasn’t quite working as I expected. It was just setting the date at the time of the daemon started up but wasn’t dynamically being set at the time the schedule was run. I’m sure I’m doing something wrong
d

daniel

03/22/2021, 3:14 PM
@dhume you want
run_config_fn
with a lambda, not run_config - that'll make it computed at execution time rather than at definition loading time
d

dhume

03/22/2021, 3:15 PM
Thanks. That makes more sense
and you can only set either
run_config
OR
run_config_fn
3 Views