Hi Team! I'd like to understand how Dagster uses a database for internal storage. Is it recommended to use external one for better reliability (instead of the one created by Helm chart)? Does Dagster make a lot of load on the database? Is it a good idea to use the same database for Dagster internal storage and for ETL jobs?

Good questions. Somewhere in the documentation, it mentions that using an external db is a good idea. I can’t speak to load, as my use-case is currently pretty small, but when I floated the idea of using an existing database instance server to a devops guy I know, he didn’t raise any concerns

I'm also curious about table name collisions...

How so?

I can't find a list of database tables that dagster creates for itself. Where does it live in documentation?

I think you should give dagster it's own database that only dagster will use. This way there will be no collisions.

I wasn’t clear in my original message, but Anatoly says it more clearly 🙂

Once you're up and running we do recommend using an external database, yeah: https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm#database - that way if you uninstall your helm chart you won't lose all your data, and you can take advantage of managed DB solutions like RDS