how do I delete the job history

As of my knowledge cutoff date in early 2023, Dagster does not provide a built-in command or function specifically for deleting job history directly through its APIs or CLI. Job history in Dagster is typically stored in an instance's event log, which is a part of the instance's underlying storage system (such as a PostgreSQL or SQLite database). If you need to delete job history, you would generally have to interact with the storage system directly. This could involve executing SQL commands to delete records from the relevant tables in the database that store the job run history. However, this is a sensitive operation and should be done with caution to avoid corrupting the instance's state. Before proceeding with any manual deletion, it's important to ensure you have proper backups of your data and understand the implications of modifying the database directly. If you're not sure how to proceed, or if the documentation has been updated with new features or methods for managing job history since my last update, I recommend checking the latest Dagster documentation or reaching out to the Dagster community for guidance. If you're facing issues with job history that you'd like to resolve, please provide more details, and I can offer debugging advice or ask follow-up questions to assist you further.

@Scout Sorry, I ment runs history. I can only delete 25 at the time in the UI

We understand that managing run history can be important for maintaining the performance and usability of your Dagster instance. As of my last update, the Dagster UI allows for bulk actions on runs, such as re-executing or terminating multiple runs at once. However, the ability to delete more than 25 runs at a time through the UI may not be directly supported. For users who need to delete a large number of runs, one approach is to use the Dagster Python API to programmatically delete runs. Here's an example script provided in the discussions that you can use as a reference to delete runs older than a certain date:

python import datetime from dagster import DagsterInstance, RunsFilter  instance = DagsterInstance.get()  # Define the time threshold for what is considered old, e.g., 1 week week_ago = datetime.datetime.now() - datetime.timedelta(days=7)  old_run_records = instance.get_run_records(     filters=RunsFilter(created_before=week_ago),     limit=10,  # Limit how many are fetched at a time, perform this operation in batches     ascending=True,  # Start from the oldest )  # In this simple example, we delete serially # For higher throughput, you could parallelize with threads for record in old_run_records:     # Delete all the database contents for this run     instance.delete_run(record.dagster_run.run_id)

This script fetches and deletes runs that were created before a certain date (one week ago in this example). You can adjust the

week_ago

variable to suit your needs. The

limit

parameter controls how many runs are fetched at a time, and you can run this script in batches to manage the deletion process. Please note that deleting runs is a destructive operation that will remove the events, tags, and run records from the database. This means that Dagster will no longer have any record of these runs, which could impact partitioned jobs and assets. Therefore, you should perform this operation with great care. If you have a large number of runs to delete and are concerned about the performance impact of deleting them one by one, you might consider running the deletion in parallel using threads or adjusting the batch size. Before running any scripts that modify your instance's data, ensure you have appropriate backups and understand the implications of these changes. If you're unsure or if there have been updates to Dagster's capabilities since my last update, please refer to the latest documentation or seek assistance from the Dagster community.