https://dagster.io/ logo
#announcements
Title
# announcements
s

schrockn

08/06/2020, 9:43 PM
Configured Users have reported that they often have large, repeated chunks of configuration across their pipelines that they never want to change at runtime. Based on this feedback, we’ve added a new ability to created pre-configured resources (as well as other definition types) that fully or partially apply configuration to pre-existing reusable resources. We imagine infrastructure teams building pre-configured resources that specify exactly the configuration they expose to leaf teams.
Copy code
from some_aws_library import s3_session

# interface to s3_session:
# @resource(config_schema={'region': str, 'use_unsigned_session': bool})
# def s3_session(_init_context):

# bake all configuration with code:
east_unsigned_s3_session = s3_session.configured(
    {'region': 'us-east-1', 'use_unsigned_session': False}
)    
    
# or allow your users only to configure the region only
@configured(s3_session, config_schema={'region': str})
def unsigned_s3_session(config):
    return {'region': config['region'], 'use_unsigned_session': False}
Configured also be applied to solid definitions. This allows one to apply configuration mapping without having to introduce a
composite_solid
layer:
Copy code
@solid(config_schema={'age': int, 'name': str})
def introduce(context):
    return "{name} is {age} years old".format(**context.solid_config)

introduce_aj = configured(introduce, name="introduce_aj")({'age': 20, 'name': "AJ"})

@pipeline
def return_int_pipeline():
    introduce_aj()
6 Views