Hello guys I m using Dagster 0 9 22 and running it inside a

Question

Hello guys I m using Dagster 0 9 22 and running it inside a container and looking to reduce the size of database because the events logs table is now over 60GB I have some questions does the CLI dagster wipe delete the entire data or can I specify a date range which tables should I drop If I only delete data from events logs will my service still work Thank you in advance and have a nice day

Brian Abelson · Answer

+1 on this im wondering if i can run a scheduled dagster job to wipe logs older than a week

schrockn · Answer

dagster run delete lt lt run id gt gt deletes a particular run and all of its event log

schrockn · Answer

In terms of getting a run by date range I m not sure we have that capability currently We have pagination APIs so one could do only keep the last 1000 runs policy or similar

schrockn · Answer

cc < prha> who is more deeply familiar with that

Brian Abelson · Answer

so theoretically one could query the underlying runs table for that list of run ids and pass it to `dagster run delete lt lt run id gt gt `

schrockn · Answer

yes

schrockn · Answer

it should be easier than that but that will work well

schrockn · Answer

just be careful undoable operation

schrockn · Answer

by it should be easier than that I mean we should have better built in support

schrockn · Answer

I think that you articulated the simplest solution possible given current system

Brian Abelson · Answer

got it thanks for clarifying crystal ball

alex · Answer

cli `dagster run list` is available to see runs

Nicolas Gaillard · Answer

Thank you for your messages I will try something and update this post Have a nice day

Brian Abelson · Answer

< alex> yeah but `dagster run list` doesn t have any flags for filtering by time of run nor does the output include this information you just get this ```Run 61338fdb 8b54 42e0 ae52 021db104b495 Pipeline mysql drupal to psql warehouse```