Is there a way to check and compare the metadata of an asset with a previous materialization of that asset? Say I want to compare the result of the num_results in a previous materialization with the current materialization.

Yes, there is a way to compare the metadata of an asset with a previous materialization of that asset in Dagster. You can access the metadata of the latest materialization and use it to compare with the current materialization's metadata. Here's an example from the documentation that demonstrates how to access the metadata of the latest materialization of an asset and use it within the implementation of the same asset:

python from dagster import asset, AssetKey  @asset def asset1(context):     instance = context.instance     materialization = instance.get_latest_materialization_event(AssetKey(["asset1"])).asset_materialization     previous_num_rows = materialization.metadata["num_rows"]      # Logic to calculate the current number of rows     current_num_rows = ...      # Compare previous_num_rows with current_num_rows     # ...      return ...

In this example,

previous_num_rows

is retrieved from the metadata of the last materialization of the asset with key

AssetKey(["asset1"])

. You can then perform any comparison or checks you need with the current materialization's

num_rows

. Please note that this example assumes that the metadata for the number of rows is stored with the key

"num_rows"

and that you have logic in place to calculate the current number of rows (

current_num_rows

). Adjust the asset key and metadata keys as needed for your specific use case.

Is it possible to provide context to the asset function, so that I can run it from outside dagster UI?

Yes, you can provide context to an asset function and run it outside of the Dagster UI using the

build_asset_context

function. This function allows you to construct an asset execution context with the necessary parameters, such as resources, config, and instance, which can then be passed to your asset function when invoking it programmatically. Here's an example from the documentation on how to use `build_asset_context`:

python from dagster import build_asset_context, asset  @asset def my_asset(context):     # Your asset logic here     return ...  # Build an asset context without any additional parameters context = build_asset_context()  # Now you can invoke your asset with the built context result = my_asset(context)

In this example,

build_asset_context

is used to create a context that is then passed to the

my_asset

function when it is invoked. If your asset requires specific resources or configuration, you can provide them as arguments to `build_asset_context`:

python context = build_asset_context(     resources={"my_resource": my_resource_instance},     asset_config={"my_config_key": "my_config_value"} )

This allows you to run your asset function in a script or an interactive environment like a Jupyter notebook, independently of the Dagster UI or Dagster's job execution machinery.