Hello, my

DAGSTER_HOME/history/runs

folder size is around 8GB, and

DAGSTER_HOME/storage

is about 1GB. It is safe to remove some of this folders? Or how can I free up some space? I would like to remove old logs, but without removing the sucess/fail state of old partitions. Thank you so much in advance!

Hi! Our recommended way for reducing the size of these databases is to occasionally delete old and unneeded records, like runs from a significant period of time ago. Here's a Github discussion that shows you how to clean up old runs https://github.com/dagster-io/dagster/discussions/12047

Hi Tim, thanks for the fast answer. But in our case 100% of the runs are partitions, so if we delete the old runs then we would have to be careful for not re-running old partitions that will be shown as "missing" after the cleanup. Is it not possible to only delete the logs, but keeping the SUCCEED status of the runs/partitions?

Hmm, there might be a better way, but after deleting, you can tell Dagster that those partition materializations already happened https://github.com/dagster-io/dagster/discussions/12561 Would that fit the bill?

Looks good, we will try it. Thanks Tim!