Manuel Fideles
03/13/2024, 3:22 PMOutput
as well as multiple AssetCheckResult
asset checks?Scout
03/13/2024, 3:22 PMOutput
and multiple AssetCheckResult
asset checks, you can define your asset using the @asset
decorator and include the checks within the body of the asset function. When the asset is materialized, you can yield the Output
for the asset's value and yield AssetCheckResult
objects for each check you want to perform. Here's a step-by-step guide based on the documentation: 1. Define your asset using the @asset
decorator. 2. Within the body of the asset function, perform the computation or transformation needed to generate the asset's value. 3. Yield an Output
object with the asset's value. 4. Perform your checks and for each check, yield an AssetCheckResult
object indicating whether the check passed or failed, and include any relevant metadata. Here is an example from the documentation that illustrates this pattern: python from dagster import asset, Output, AssetCheckResult @asset def my_asset(): # Perform computation to generate asset's value asset_value = ... # Yield the asset's value as an Output yield Output(asset_value) # Perform checks and yield AssetCheckResult for each check check_passed = ... # Your check logic here yield AssetCheckResult( passed=check_passed, check_name='my_check_name', # A name for the check metadata={'details': 'Additional information about the check result'} ) # You can yield multiple AssetCheckResult objects for different checks another_check_passed = ... # Another check logic here yield AssetCheckResult( passed=another_check_passed, check_name='another_check_name', metadata={'details': 'Additional information about this check result'} )
In this example, my_asset
is an asset function that yields both the asset's value and the results of multiple checks. Each AssetCheckResult
is yielded separately, and you can include as many checks as needed. Remember that the AssetCheckResult
objects will be associated with the asset being materialized, and the results of these checks will be visible in the Dagster UI, allowing you to monitor the data quality and integrity of your assets.Manuel Fideles
03/13/2024, 3:28 PMScout
03/13/2024, 3:28 PMOutput
and multiple AssetCheckResult
asset checks using pytest, you can invoke the asset function directly in your test function, as assets are regular Python functions. You can then capture the yielded values and AssetCheckResult
objects to assert that they meet your expectations. Here's how you can structure your pytest test function: 1. Call the asset function directly within your test function. 2. Use a loop or list comprehension to collect all the yielded results from the asset function. 3. Assert that the Output
value is as expected. 4. Assert that each AssetCheckResult
indicates the checks have passed and that the metadata is correct. Here's an example of how you might write a pytest test for the asset defined in the previous example: python import pytest from dagster import execute_in_process from my_assets_module import my_asset def test_my_asset(): # Execute the asset function and collect the yielded results results = list(my_asset()) # Separate the Output and AssetCheckResult objects output_results = [r for r in results if isinstance(r, Output)] check_results = [r for r in results if isinstance(r, AssetCheckResult)] # Assert that the Output value is as expected assert output_results[0].value == expected_value # Assert that all checks have passed and metadata is correct for check_result in check_results: assert check_result.passed assert check_result.metadata['details'] == 'Expected metadata details'
In this test, expected_value
should be replaced with the actual value you expect the asset to produce. The AssetCheckResult
objects are asserted to ensure that each check has passed and that the metadata matches what you expect. Remember to import the necessary classes and functions from Dagster and your asset module. Adjust the assertions according to the specific logic and metadata of your asset checks. By directly invoking the asset function and asserting on the results, you can effectively test both the output value and the asset checks in isolation from the rest of your Dagster pipeline.