clay
04/18/2023, 1:52 PM@assets
can? For example, this works fine:
@asset(required_resource_keys={"snowflake"})
def my_thing(context: OpExecutionContext) -> None:
return
but this does not work, even when right next to assets
@op(required_resource_keys={"snowflake"})
def my_thing(context: OpExecutionContext) -> None:
return
and the error is
dagster._core.errors.DagsterInvalidDefinitionError: resource with key 'snowflake' required by op 'my_thing' was not provided. Please provide a <class 'dagster._core.definitions.resource_definition.ResourceDefinition'> to key 'snowflake', or change the required key to one of the following keys which points to an <class 'dagster._core.definitions.resource_definition.ResourceDefinition'>: ['io_manager']
Le Yang
04/18/2023, 2:02 PM@op
is not independently usable i.e. it needs to be utilized from a @job
which in turn provide the resource definitions.
Look at your Definition, the assets are included but ops are not. If assets aren't included in the Definition they wouldn't see the configured resources either.clay
04/18/2023, 2:45 PM@op(required_resource_keys={"snowflake"})
def my_thing(context: OpExecutionContext) -> None:
return
@job(required_resource_keys={"snowflake"})
def my_job():
my_thing()
and then @job will expose the resource to the @op?Le Yang
04/18/2023, 2:54 PM@job
decorator similar to how you would supply them to a Definition, like this:
@op(required_resource_keys={"snowflake"})
def my_thing(context: OpExecutionContext) -> None:
return
@job(resource_defs={"snowflake": snowflake_io_manager...})
def my_job():
my_thing()
clay
04/18/2023, 2:55 PMLe Yang
04/18/2023, 2:56 PMDefinition()
or @job()