I’m looking for some clarification on how configs work.
I have an asset that defines two config parameters via the config_schema. I then create a schedule, which has a job using ‘define_asset_job’ which has a config that is ultimately loaded from a yml file on disk. I then have a repository.py file that I load into dagster.
When my schedule executes, I am expecting the schedule to execute the job, which will load the yml from disk and materialize my asset. However, the schedule errors stating that there is no config. Am I correct in expecting the yml to be loaded from disk? Or does dagster intervene and expect the config to be supplied via the UI?
:dagster-bot-resolve: 1
s
sean
10/31/2022, 8:09 PM
Hi Robert, Where exactly are you specifying that the config comes from this yml file?
Just reading the description of what you’re trying to accomplish, I would expect you to load the yml file inside the schedule function and pass its contents under the`run_config` key of the
RunRequest
returned from the schedule function, as shown here.
r
Robert Wade
11/03/2022, 1:59 AM
@sean Thank you for the reply and the pointer to the RunRequest. I was able to accomplish what I was trying to do. What is difficult for someone new to Dagster is that there appears to be numerous ways to accomplish things and it isn't completely obvious when to use each method. I appreciate you taking the time to point me in the right direction.
s
sean
11/03/2022, 1:40 PM
Thanks for the feedback, we’re actively working to simplify Dagster’s many abstractions and make the new user experience easier!