https://dagster.io/ logo
#ask-ai
Title
# ask-ai
a

Abhishek Tripathi

02/24/2024, 10:12 AM
Passing configuration for downstream assets
s

Scout

02/24/2024, 10:12 AM
In Dagster, you can pass configuration values to assets at runtime, which is particularly useful for downstream assets that may depend on dynamic values or external parameters. According to the documentation, assets in Dagster can specify a config schema, allowing you to provide values to assets at runtime. This is done by defining a config model subclass of
Config
and specifying a
config
parameter for the asset's function. Here's an example from the documentation that demonstrates how to define and use configuration for a downstream asset:
Copy code
python from dagster import Config, asset  @asset def my_upstream_asset() -> int:     return 5  class MyDownstreamAssetConfig(Config):     api_endpoint: str  @asset def my_downstream_asset(config: MyDownstreamAssetConfig, my_upstream_asset: int) -> int:     data = requests.get(f"{config.api_endpoint}/data").json()     return data["value"] + my_upstream_asset
In this example,
my_downstream_asset
is a downstream asset that depends on
my_upstream_asset
. The
MyDownstreamAssetConfig
class defines the configuration schema for
my_downstream_asset
, which includes an
api_endpoint
parameter. When materializing
my_downstream_asset
, you can provide the
api_endpoint
configuration value at runtime. When launching a job that materializes assets, you can provide run configuration for each entity. The provided run configuration values correspond to a configuration schema attached to the asset definition. Dagster validates the run configuration against the schema and proceeds only if validation is successful. If you have further questions or need assistance with a specific use case, feel free to ask!
4 Views