Rasmus Bonnevie
02/23/2023, 1:16 PMfrom dagster import AssetObservation, op
from dagster import AssetObservation, op
@op
def observation_op(context):
df = read_df()
context.log_event(
AssetObservation(asset_key="observation_asset", metadata={"num_rows": len(df)})
)
return 5
where the asset is loaded inside the op, but this goes against most of your own recommendations about abstracting loading logic away in IO managers which are then not known/instantiated until Definitions
is instantiated.Rasmus Bonnevie
02/23/2023, 2:34 PMmaterialized_to_memory
which starts with a data_source
asset with hard-coded loading, side-stepping cases involving e.g. SourceAsset
where you are reliant on an IO manager.
https://docs.dagster.io/concepts/testing#testing-multiple-software-defined-assets-togetherTim Castillo
02/23/2023, 3:34 PMRasmus Bonnevie
02/23/2023, 3:58 PM