https://dagster.io/ logo
#ask-community
Title
# ask-community
s

stefan hansan

02/16/2023, 6:50 PM
Hello! I have a (hopefully) quick question. I want to materialize an asset on a recurring basis so I am using define_asset_job in conjunction with ScheduleDefinition. My asset requires some resources like a db cursor and requests session. How can I pass these resources to it using define_asset_job?
👀 1
o

owen

02/16/2023, 7:05 PM
hi @stefan hansan! if you're using Definitions, then you can just supply those resources there, i.e.
Copy code
my_job = define_asset_job("my_job", selection=AssetSelection.keys("my_asset")

my_schedule = ScheduleDefinition(job=my_job, cron_schedule="...")

defs = Definitions(
    assets=[my_asset],
    jobs=[my_job],
    schedules=[my_schedule],
    resources={"db_cursor": db_cursor, "requests": requests_session},
)
s

stefan hansan

02/16/2023, 7:19 PM
Thanks for the response! What if I wanted to pass in a config_schema to the asset?
o

owen

02/16/2023, 7:20 PM
if your asset has a config schema, then you can pass in values for that config when calling define_asset_job (i.e.
define_asset_job(..., config={"ops": {"my_asset": {"config": {"my_key": 1, "other_key": "value"}}}})
s

stefan hansan

02/16/2023, 7:25 PM
I see! As a follow up, how should i wire my unit test class to launch the job? afaik, i have to provide both the resources and config data in the execute_in_process function call.
o

owen

02/16/2023, 7:29 PM
I would use
materialize_to_memory
instead of
execute_in_process
for this use case. This has both a run_config and resources argument (so you get the added bonus of being able to mock out those resources). This would look like
Copy code
from dagster import materialize_to_memory

result = materialize_to_memory([my_asset], resources=..., run_config=...)
s

stefan hansan

02/16/2023, 7:43 PM
I am getting an error, "resource with key 'requests_session' required by op x was not provided." I am assuming I am not passing it correctly to my asset. is the following the correct way to feed the asset the resources?
Copy code
result = materialize_to_memory([my_asset], resources={'resources': { {'requests_session': {'config': {'CERT': {'env': CERT}}}}}}
thank you for your help btw!
o

owen

02/16/2023, 7:49 PM
the shape of your "resources" dictionary is a bit off -- the keys should just be the resource key and the value should be the resource object
s

stefan hansan

02/16/2023, 8:01 PM
so it would be
Copy code
resources= {'requests_session': requests_session_configured}
I am still getting the error 'resource with key, "requests_session" required by op x not provided' even though i think i am providing it with the above snippet
o

owen

02/16/2023, 9:22 PM
do you mind sharing the full code?
18 Views