https://dagster.io/ logo
#ask-community
Title
# ask-community
g

Gabriel Montañola

02/23/2024, 3:10 PM
Hi there folks! Questions for those who are running Dagster OSS in production 1. Which database are you using for *RunStorage? (*RDS PostgreSQL, on-premises MySQL, etc.) 2. How about the specs? (CPU and Memory) 👉 We're running a RDS Postgres
t4g.small
with 2 vCPU and 2GiB memory here but the performance degrades quickly and we constantly need to delete runs, ticks and tags from the database. 🗣️ Dagster evolved from a POC here a year ago, but the database never evolved from "POC status". Now it's upgrade time and community feedback is much appreciated.
dagster spin 1
c

Chris Roth

02/26/2024, 3:12 AM
I'll check our production settings tomorrow and reply. We're kicking off several thousand runs a day without issue using partitions.
dagster yay 1
We're running our prod setup on a $112 USD / mo cloud hosted kubernetes cluster, so we have ~100x more memory than you've noted above. But this cluster also does most of the compute that's launched by the Dagster instance as well.
👍 1
g

Gabriel Montañola

02/26/2024, 5:46 PM
thanks cris! so you're running the database @ k8s too, right?
c

Chris Roth

02/26/2024, 5:48 PM
We're using a DO managed Postgres database: https://www.digitalocean.com/products/managed-databases
But, a lighter cost alternative would be to run postgres in the cluster connected to a decent block-storage persistent volume.
g

Gabriel Montañola

02/26/2024, 5:51 PM
yeap! and what are the specs of the managed PG db?
> But, a lighter cost alternative would be to run postgres in the cluster connected to a decent block-storage persistent volume. things are getting nicer for these use cases (https://kubeblocks.io/) but I prefer the peace of mind of a managed db if you have a small team.
c

Chris Roth

02/26/2024, 5:56 PM
I agree with the peace of mind of a managed DB, especially when you upgrade your containers and just hope that Postgres reconnects to the right PVC. I mean everything should work, but who wants the anxiety?
We spend $100/mo on our managed DB, but that's running everything, not just our Dagster instance.
👍 1
Are you selling vegetables?
g

Gabriel Montañola

02/26/2024, 5:58 PM
Thanks a lot for the feedback, @Chris Roth!
Are you selling vegetables?
plenty of them, indeed 😄
but b2b and b2bc, not directly to the customer.
c

Chris Roth

02/26/2024, 5:59 PM
Nice, spain, portugal, brazil?
My language recognition isn't 100%.
g

Gabriel Montañola

02/26/2024, 6:00 PM
Brazil!
c

Chris Roth

02/26/2024, 6:01 PM
Awesome. Keep it up! Getting real food on people's tables is noble work.
daggy love 1