With the new Pythonic Config system using Pydantic, what is the best way to configure a set of resource/iomanagers from a yaml? Use the old method or use Pydantic from yaml library or does dagster provide its own method?

Hi Chris, you can still pass

run_config

in the old format (directly loaded from YAML). This is unlikely to go away anytime soon because this is also how config needs to come in from non-Python APIs (CLI, dagit)

Hi, Im also interested in this. Looking at migrating my legacy resources to the new pythonic resources as I specifically want to use resources in sensors to query a database. However, the way I currently do it, is provide the connection details for the config for a resource at run time in a schedule using a yaml file. Do you need to add resources to the

Definitions

using this new way? or, just leave them out and still be able to pass in a yaml file to

run_config

in the

RunRequest

in a schedule or sensor?

@Megan Beckett I am sorry I missed your question. I have not yet coded this fully but yes it would go along with the sensor. Here is something to look at:

def make__sensor(sensor_job, tables_partitions_def, prefix=None, run_config=None) -> SensorDefinition:

name = tables_partitions_def.name.replace('-','_')

config = {"ops": {f'{name}': {"config": { "file_url": ""} } } }

if run_config:

config = deep_merge_dicts(config, run_config)

url_config = config['ops'][f'{name}']['config']

@sensor(job=sensor_job, name=f'{name}_sensor')

def sensor(context: SensorEvaluationContext,

sensor_resource: SensorResource):

...

run_requests = [RunRequest(... run_config=update_file_url(config, url_config,f'construct_url_w_{key}')) for key in keys]

...

return SensorResult(run_requests=run_requests, dynamic_partitions_requests=[tables_partitions_def.build_add_request(new_keys)])

return sensor