Linton Ye
02/18/2024, 7:36 PMScout
02/18/2024, 7:36 PM@asset
decorator. If you have an asset that requires a specific configuration, you can define a config schema for that asset. 2. Use the define_asset_job
function to create a job that targets all assets. By default, define_asset_job
without a selection argument will target all assets in the code location. 3. To forward the config to a specific asset, you can use the config
parameter within the asset definition to access the run configuration provided at runtime. Here is an example that illustrates these steps: python from dagster import asset, define_asset_job, Field, ConfigMapping # Define an asset that requires configuration @asset(config_schema={"param": Field(str)}) def asset_requires_config(context): # Access the config provided at runtime param_value = context.op_config["param"] # Your asset logic here ... # Define other assets @asset def another_asset(): # Your asset logic here ... # Define a job that targets all assets all_assets_job = define_asset_job( name="all_assets_job", config={ "ops": { "asset_requires_config": { "config": { "param": "value_to_forward" } } } } ) # Include the job in your repository
In this example, asset_requires_config
is an asset that requires a configuration parameter param
. The all_assets_job
is defined to target all assets, and it includes a run configuration that specifies the value for param
that should be forwarded to asset_requires_config
. Please note that the config
parameter in define_asset_job
is used to provide a default run configuration for the job. This configuration will be viewable and editable in the Dagster UI. Be careful with secrets since they will be exposed in the UI. Keep in mind that the exact syntax and structure may vary depending on the version of Dagster you are using and the specific requirements of your assets and job. Always refer to the latest Dagster documentation for the most accurate and up-to-date information.