Ruoyu Qian
10/13/2022, 7:22 PMrequired_resource_keys={"hightouch"}
and a job that call the op with _resource_defs_={_"resources"_:{_"hightouch"_: hightouch_resource}}
But I got an error like this , what might be the issue, is it syntax issue?
dagster._check.CheckError: Value in Mapping mismatches expected type for key resources. Expected value of type <class 'dagster._core.definitions.resource_definition.ResourceDefinition'>. Got value {'hightouch': <dagster._core.definitions.resource_definition.ResourceDefinition object at 0x146eb8c40>} of type <class 'dict'>.
# My resources
class Hightouch:
_endpoint = "some endpoint"
def __init__(self, api_key: str):
self,
self.api_key = api_key
@property
def headers(self):
return {"Authorization": f"Bearer {self.api_key}"}
def start_sync(self, sync_id: int) -> int:
response = <http://requests.post|requests.post>(
self._endpoint + f"{sync_id}/trigger", headers=self.headers
)
return response
@resource(
config_schema={
"HIGHTOUCH_API_KEY": Field(StringSource, is_required=True),
}
)
def hightouch_resource(context) -> Hightouch:
return Hightouch(
api_key=context.resource_config["HIGHTOUCH_API_KEY"],
)
hightouch_configured = hightouch_resource.configured(
{
"HIGHTOUCH_API_KEY": os.getenv("HIGHTOUCH_API_KEY"),
}
)
# My ops
from resources.hightouch import *
@op(required_resource_keys={"hightouch"})
def sync_user(context):
"""
This sync function only syncs user on Hightouch
"""
syncs = context.resources.hightouch
syncs.start_sync()
#My jobs
@job(
description="""
This job runs the op
""",
tags={"ecs/cpu": "2048", "ecs/memory": "8192"},
resource_defs={"resources":{"hightouch": hightouch_resource}},
config=some_config,
)
def my_job():
sync_user()
Adam Bloom
10/13/2022, 7:26 PM{"resources":{"hightouch": hightouch_configured}}
Ruoyu Qian
10/13/2022, 7:44 PMAdam Bloom
10/13/2022, 7:45 PMRuoyu Qian
10/13/2022, 7:51 PMGot value {'hightouch': <dagster._core.definitions.resource_definition.ResourceDefinition object at 0x15c738b20>} of type <class 'dict'>.
was there anything wrong with my resource definition? the error is saying it received an ResourceDefinition
, but isn’t that expected?Adam Bloom
10/13/2022, 8:05 PMresource_defs={"hightouch": hightouch_resource}
Ruoyu Qian
10/13/2022, 8:24 PMsaravan kumar
11/04/2022, 4:56 AM