is there a way to combine presets with e.g. the config generated from a schedule or partition, so you don't have to define secret envs in multiple places for the same pipeline?

Hi Klaus- does the example here of bringing in a preset into a schedule function answer your question? https://github.com/dagster-io/dagster/discussions/2943

this might be it... but I don't understand how the date from the schedule is passed into the preset

Presets don't take in inputs, so if that's a requirement that's probably not the right thing to use here. Is there a benefit to using a preset vs. having a shared helper function that each of the schedules call to generate config?

sorry, I probably expressed it wrong: In the scheduler and partitions I want to take the preset as the base configuration and then add the from date and to date dynamically. Is this possible with above method?

Doing this is possible, yeah - the preset can supply a run config dict, and then in the schedule function you can get it and then modify it however you like.

Just curious on this. I tried to use a dynamic date within a schedule

demo_schedule = ScheduleDefinition(
    name="DemoSchedule",
    cron_schedule="*/2 * * * *",
    pipeline_name="demo_pipeline",
    run_config={
        "solids": {
            "second_power": {
                "inputs": {
                    "date": {
                        "value": datetime.datetime.now().strftime(
                            "%m/%d/%Y, %H:%M:%S"
                        )
                    },
                }
            },
        },
    },
    mode="dev",
)

And it wasn’t quite working as I expected. It was just setting the date at the time of the daemon started up but wasn’t dynamically being set at the time the schedule was run. I’m sure I’m doing something wrong

@dhume you want

run_config_fn

with a lambda, not run_config - that'll make it computed at execution time rather than at definition loading time

Thanks. That makes more sense