https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Alex Bichler

02/28/2024, 6:51 PM
Hey folks, I added 90 day data retention to our dagster instance per the documentation found here (https://docs.dagster.io/deployment/dagster-instance#data-retention). I expect that this would delete entries in the some of the tables (job_ticks, runs, event_logs, etc..) in the database that are older than 90 days. Are there any additional configurations or jobs that need to be scheduled to kick off the purges? What is the default schedule for the purges?
p

Pedram Navid

02/29/2024, 5:01 AM
The docs state "schedule/sensor tick data" only, not runs or event logs.
a

Alex Bichler

02/29/2024, 5:48 PM
I'm not sure I understand what "schedule/sensor tick data" is. Does this mean that the data in the job_ticks table should be deleted after it's aged 90 days? I still see entries well over a year old in this table. Maybe this is a bug? Or, is this purging process not retroactive?
p

Pedram Navid

02/29/2024, 5:53 PM
did you restart the dagster instance? I think this gets picked up by the instance on load
i think it will only start purging if the sensor is still active and running too if im understanding the code correctly.
a

Alex Bichler

02/29/2024, 6:01 PM
We're hosting on a fargate cluster, the image was rebuilt on deployment. I did confirm that the retention is defined in the configuration under deployment. All the schedules and sensors are toggled on as well. I'm going to toggle them all off and on.
👍 1
16 Views