Hi folks I m having trouble with `dagster asset wipe` I have dagster #ask-community

Hi folks, I'm having trouble with `dagster asset w...

Eric Brown

08/10/2023, 6:48 PM

Hi folks, I'm having trouble with

dagster asset wipe

. I have an asset the refreshing of which depends on metadata from the previous run. It has special logic to handle the first time the asset is refreshed when presumably there shouldn't be any metadata from the previous run. I'm using

dagster asset wipe

, and I can see in the UI that the asset appears to have been wiped (no metatdata), but when I run the refresh, the run is still able to access metadata from the previous run (which was supposed to have been wiped). See code in 🧵

🤖 1

Eric Brown

08/10/2023, 6:49 PM

Copy code

@asset
def my_asset(context):
    instance = context.instance
    latest_materialization_event = instance.get_latest_materialization_event(AssetKey(["my_asset"]))

    if latest_materialization_event is None:
        print('this is the first materialization event')
    else:
        print('this is not the first materialization event')

Eric Brown

08/10/2023, 6:55 PM

Materialize the asset for the first time, we see

Copy code

this is the first materialization event

Run it again, we see

Copy code

this is not the first materialization event

then, run

Copy code

dagster asset wipe my_asset

we no events in the UI which seems to confirm that the events have been wiped. Refreshing the asset, we would expect to see "this is the first materialization event", but instead we get "this is not the first materialization event" I'm running locally using dagster 1.4.4, using filesystem for run storage

alex

08/10/2023, 9:52 PM

the “asset wipe” is currently a soft delete and it looks like

get_latest_materialization_event

ignores the soft delete metadata would you be up for filing an issue?

Eric Brown

08/11/2023, 2:25 AM

https://github.com/dagster-io/dagster/issues/15806

thank you box 1

Open in Slack

Previous Next