https://dagster.io/ logo
Join Slack
Powered by
hi team, was wondering how the `resources` in `Run...
# ask-community
j
hi team, was wondering how the 
resources
in 
RunConfig
is supposed to work, detail in 🧵
Copy code
from dagster import ConfigurableResource, RunConfig, job, op


class DummyResource(ConfigurableResource):
    foo: str = "foo"


@op
def greet_dummy(dummy: DummyResource) -> None:
    print(f"Hi {dummy.foo}?")


@job
def greet_dummy_job() -> None:
    greet_dummy()


# this does not work
op_result = greet_dummy_job.execute_in_process(run_config=RunConfig(resources={"dummy": DummyResource()}))

# this works
op_result = greet_dummy_job.execute_in_process(run_config=RunConfig(), resources={"dummy": DummyResource()})
i thought we could just specify resources as part of 
RunConfig
but the job doesn't appear to be able to find the config properly
this is running on 
dagster-1.4.2
a
Hi Joey! Afaik, that way of passing resources in the function parameters only works with assets. With ops you have to pass a set of _required_resource_keys_ when defining your op.
Copy code
@op(required_resource_keys={'dummy'})
def greet_dummy(context) -> None:
    dummy = context.resources.dummy
    print(f"Hi {dummy.foo}?")
I'm just a fellow user though, someone from elementl can correct me if I'm wrong.
j
ty Auster! although the above usage is acutally legal, if you check https://docs.dagster.io/concepts/resources, switch to 
Using Ops and jobs
followup, I used to test multiprocess runs in local script, but no longer works using 
RunConfig
(used to be a single dict and resources used to be defined via schema and 
@resource
decorator)
Copy code
instance = DagsterInstance.get()
execute_job(
    reconstructable(reconstructable_dummy_job),
    instance=instance,
    run_config=RunConfig(
        resources={"dummy": DummyResource()}, execution={"config": {"multiprocess": {"max_concurrent": 2}}}
    ),
)
a
oh my bad, apologies for butting in 😅 I'll leave it to the pros
😁 1
o
Hi @Joey! This is definitely a bit confusing, but the 
resources
parameter for the 
RunConfig
object refers to configuration for your resources, not fully-instantiated resources themselves. Instead, you'd want to pass in that 
resources={"dummy": DummyResource()}
into the 
execute_job
/ 
execute_in_process
command directly, i.e. 
...execute_in_process(..., run_config=RunConfig(execution={...}), resources={...})
j
thanks owen! correct me if wrong, but execute_job doesn't seem to be exposing resources as one of the kwargs, is there a different way to pass it in? Also, the RunConfig resource kwarg is essentially not used?
o
ah that's my bad -- you'd have to explicitly supply those resources when doing graph.to_job(resources=...), or 
@job(resources=...)
. The reason for the arg on the RunConfig object is to configure those resources that have already been bound to the job. You can imagine having some resource that reads from a certain API, and then configuring it with specific credentials via the RunConfig
❤️ 1
Open in Slack
PreviousNext
Powered by