https://dagster.io/ logo
Join Slack
Powered by
Hello guys, I'm using Dagster 0.9.22 (and running ...
# announcements
n
Hello guys, I'm using Dagster 0.9.22 (and running it inside a container) and looking to reduce the size of database because the "events_logs" table is now over 60GB. I have some questions: • does the CLI "dagster wipe" delete the entire data or can I specify a date range? • which tables should I drop? If I only delete data from "events_logs" will my service still work? Thank you in advance and have a nice day!
b
+1 on this. im wondering if i can run a scheduled dagster job to wipe logs older than a week.
s
dagster run delete <<run_id>> deletes a particular run and all of its event log
In terms of getting a run by date range, I’m not sure we have that capability currently. We have pagination APIs so one could do “only keep the last 1000 runs” policy or similar
cc: @prha who is more deeply familiar with that
b
so theoretically one could query the underlying runs table for that list of run ids and pass it to 
dagster run delete <<run_id>>
s
yes
it should be easier than that but that will work well
just be careful! undoable operation
by “it should be easier than that” I mean we should have better built-in support
I think that you articulated the simplest solution possible given current system
b
got it! thanks for clarifying 🔮
a
cli 
dagster run list
is available to see runs
n
Thank you for your messages, I will try something and update this post ! Have a nice day
b
@alex yeah, but 
dagster run list
doesn't have any flags for filtering by time of run, nor does the output include this information. you just get this:
Copy code
Run: 61338fdb-8b54-42e0-ae52-021db104b495
     Pipeline: mysql_drupal_to_psql_warehouse
Open in Slack
PreviousNext
Powered by