Hi team, in an asset sensor with dynamic partition...
# ask-community
Hi team, in an asset sensor with dynamic partitions, I receive an error
After dynamic partitions requests are applied, it does not exist in the set of valid partition keys.
Is there a way to clear the partitions from the database? I'm using dagster-cloud
there was an issue with the upstream and we had to do a resync of files, meaning that previous partitions were deleted
FWIW, i tried the following to work around this:
Copy code
clear_partitions = os.environ.get("CLEAR_PARTITIONS")

    if clear_partitions == "True":
        dynamic_partitions = context.instance.get_dynamic_partitions(partitions_def.name)

        <http://context.log.info|context.log.info>(f"Clearing {len(dynamic_partitions)} partitions")

        for partition in dynamic_partitions:
            context.instance.delete_dynamic_partition(partitions_def.name, partition)

        return SkipReason(f"Skipping sensor run as partitions have been cleared")
similar fashion, my issue is due to the fact that partitions are no longer available (deleted s3 keys which are used as partition keys). so my asset_sensor was choking on an error. needed to find a way to remove all the partitions. seems like wipe assets could be cleaner...
Hi YH. Your workaround in the asset sensor makes sense. You could optionally have an op that deletes all of the partitions in the dynamic partitions def. I agree that it would be nice to enable deleting dynamic partitions in the UI. We have an issue for tracking here: https://github.com/dagster-io/dagster/issues/14026 Since dynamic partitions are a definition object that can be applied to many assets, I think it makes sense for wiping assets and wiping the dynamic partitions def to be independent actions.
hi @claire thanks for the suggestion to have an op instead. that would be a cleaner workaround
🌈 1
have upvoted the the issue 🙂