Giacomo Colombo
03/07/2024, 11:44 AMclass YearContainer:
# Mutable container for your resource value
def __init__(self, year):
self.year = year
@resource(config_schema={'env_var_name': Field(str, is_required=True)})
def custom_year_resource(init_context):
env_var_name = init_context.resource_config['env_var_name']
curr_year = int(os.getenv(env_var_name, "1990"))
return YearContainer(curr_year)
@asset(
required_resource_keys={'custom_year_resource_3'},
)
def my_asset_1(context):
curr_year = context.resources.custom_year_resource_3.year
<http://context.log.info|context.log.info>(f'this is the year {curr_year}')
# do something
Scout
03/07/2024, 11:44 AMcustom_year_resource
in an asset, you need to define the resource and then associate it with the asset by specifying the required_resource_keys
. You've already defined the YearContainer
class and the custom_year_resource
resource function correctly. However, there are a few adjustments needed in your asset definition to ensure that the resource is used properly. Here's how you can adjust your asset definition: 1. Make sure that the key you use in required_resource_keys
matches the key you assign to the resource when you build your job or repository. 2. Use the correct resource key within the asset function to access the resource. 3. Ensure that you have imported all necessary modules and functions, such as os
, resource
, Field
, and asset
. Here is an example of how you might define your asset, assuming that you have set up your resource correctly and that the resource key is `custom_year_resource`: python from dagster import asset, resource, Field import os class YearContainer: # Mutable container for your resource value def __init__(self, year): self.year = year @resource(config_schema={'env_var_name': Field(str, is_required=True)}) def custom_year_resource(init_context): env_var_name = init_context.resource_config['env_var_name'] curr_year = int(os.getenv(env_var_name, "1990")) return YearContainer(curr_year) @asset(required_resource_keys={'custom_year_resource'}) def my_asset_1(context): curr_year = context.resources.custom_year_resource.year <http://context.log.info|context.log.info>(f'this is the year {curr_year}') # do something
Please note that in the @asset
decorator, the required_resource_keys
is set to {'custom_year_resource'}
which should match the key you use when you provide the resource in your job or repository definition. When you build your job or repository, you will need to provide the resource definition and its configuration. Here's an example of how you might do that: python from dagster import job @job(resource_defs={'custom_year_resource': custom_year_resource}) def my_job(): my_asset_1() # You would then configure the resource when you launch the job, for example: # my_job.execute_in_process(run_config={'resources': {'custom_year_resource': {'config': {'env_var_name': 'YEAR_ENV_VAR'}}}})
In this example, my_job
is a job that includes the asset my_asset_1
, and it specifies the resource definition for custom_year_resource
. The run_config
is where you would provide the actual environment variable name that contains the year value.