Hello! I have a (hopefully) quick question. I want to materialize an asset on a recurring basis so I am using define_asset_job in conjunction with ScheduleDefinition. My asset requires some resources like a db cursor and requests session. How can I pass these resources to it using define_asset_job?

hi @stefan hansan! if you're using Definitions, then you can just supply those resources there, i.e.

my_job = define_asset_job("my_job", selection=AssetSelection.keys("my_asset")

my_schedule = ScheduleDefinition(job=my_job, cron_schedule="...")

defs = Definitions(
    assets=[my_asset],
    jobs=[my_job],
    schedules=[my_schedule],
    resources={"db_cursor": db_cursor, "requests": requests_session},
)

Thanks for the response! What if I wanted to pass in a config_schema to the asset?

if your asset has a config schema, then you can pass in values for that config when calling define_asset_job (i.e.

define_asset_job(..., config={"ops": {"my_asset": {"config": {"my_key": 1, "other_key": "value"}}}})

I see! As a follow up, how should i wire my unit test class to launch the job? afaik, i have to provide both the resources and config data in the execute_in_process function call.

I would use

materialize_to_memory

instead of

execute_in_process

for this use case. This has both a run_config and resources argument (so you get the added bonus of being able to mock out those resources). This would look like

from dagster import materialize_to_memory

result = materialize_to_memory([my_asset], resources=..., run_config=...)

I am getting an error, "resource with key 'requests_session' required by op x was not provided." I am assuming I am not passing it correctly to my asset. is the following the correct way to feed the asset the resources?

result = materialize_to_memory([my_asset], resources={'resources': { {'requests_session': {'config': {'CERT': {'env': CERT}}}}}}

the shape of your "resources" dictionary is a bit off -- the keys should just be the resource key and the value should be the resource object

so it would be

resources= {'requests_session': requests_session_configured}

I am still getting the error 'resource with key, "requests_session" required by op x not provided' even though i think i am providing it with the above snippet

do you mind sharing the full code?