I was interested if it was possible to store event logs in S3 or Google Cloud Storage ? For example instead of storing a SQLite database locally, just store in some bucket?

Hi Alfie - currently, the Dagster web server needs to be able to query the event log to be able to serve many of the web pages, so it needs to be backed by a SQL database

Hi Sandy, that makes sense. Thanks for the help 🙂

I’m currently trying out using Timescale DB which can offload old data to S3 but it keeps it queryable through Postgres. There are a few rough spots that I’ve run into so far, but it seems like a promising option to deal with the volume of event logs.