https://dagster.io/ logo
Title
r

Ruoyu Qian

10/13/2022, 7:22 PM
Hi dagster people, I have made a resource that contains a class which just initiate an object that can post an api request to a Hightouch endpoint. I have a op that reference the resource with
required_resource_keys={"hightouch"}
and a job that call the op with
_resource_defs_={_"resources"_:{_"hightouch"_: hightouch_resource}}
But I got an error like this , what might be the issue, is it syntax issue?
dagster._check.CheckError: Value in Mapping mismatches expected type for key resources. Expected value of type <class 'dagster._core.definitions.resource_definition.ResourceDefinition'>. Got value {'hightouch': <dagster._core.definitions.resource_definition.ResourceDefinition object at 0x146eb8c40>} of type <class 'dict'>.
# My resources
class Hightouch:
    _endpoint = "some endpoint"
    def __init__(self, api_key: str):
        self,
        self.api_key = api_key
   
    @property
    def headers(self):
        return {"Authorization": f"Bearer {self.api_key}"}

    def start_sync(self, sync_id: int) -> int:

        response = <http://requests.post|requests.post>(
            self._endpoint + f"{sync_id}/trigger", headers=self.headers
        )
        return response

@resource(
    config_schema={
        "HIGHTOUCH_API_KEY": Field(StringSource, is_required=True),
    }
)
def hightouch_resource(context) -> Hightouch:
    return Hightouch(
        api_key=context.resource_config["HIGHTOUCH_API_KEY"],
    )
hightouch_configured = hightouch_resource.configured(
    {
        "HIGHTOUCH_API_KEY": os.getenv("HIGHTOUCH_API_KEY"),
    }
)
# My ops
from resources.hightouch import *

@op(required_resource_keys={"hightouch"})
def sync_user(context):
    """
    This sync function only syncs user on Hightouch
    """
    syncs = context.resources.hightouch
    syncs.start_sync()
#My jobs
@job(
    description="""
        This job runs the op
    """,
    tags={"ecs/cpu": "2048", "ecs/memory": "8192"},
    resource_defs={"resources":{"hightouch": hightouch_resource}},
    config=some_config,
    
)
def my_job():
    sync_user()
:dagster-bot-responded-by-community: 1
a

Adam Bloom

10/13/2022, 7:26 PM
from a quick glance, I believe you need to pass the configured resource
{"resources":{"hightouch": hightouch_configured}}
r

Ruoyu Qian

10/13/2022, 7:44 PM
Assign to _`resource_defs`_ ? do I need to change the config in the job ?
a

Adam Bloom

10/13/2022, 7:45 PM
assign to resource_defs, correct. I only glanced, but the rest looks similar to my resource definitions/usage. You just need to pass configured resources to jobs
r

Ruoyu Qian

10/13/2022, 7:51 PM
hmm, error remain the same,
Got value {'hightouch': <dagster._core.definitions.resource_definition.ResourceDefinition object at 0x15c738b20>} of type <class 'dict'>.
was there anything wrong with my resource definition? the error is saying it received an
ResourceDefinition
, but isn’t that expected?
a

Adam Bloom

10/13/2022, 8:05 PM
ahh! sorry, I missed this first pass.
resource_defs={"hightouch": hightouch_resource}
looks like you had an extra level of nesting floating around
r

Ruoyu Qian

10/13/2022, 8:24 PM
Ah I see, it seems to work now Thank you very much
s

saravan kumar

11/04/2022, 4:56 AM
Hey @Ruoyu Qian wondering if ur project open source? just looking for samples...