https://dagster.io/ logo
Title
m

Matthew Whiteside

10/18/2022, 4:05 PM
Just starting out. I have a question about moving to persistent storage using postgres. I would like to use postgres for everything (event logs, schedule, run etc). In this documentation: https://docs.dagster.io/deployment/dagster-instance#dagster-storage It lists the dagster.conf config as:
storage:
  postgres:
    postgres_db:
      username: { DAGSTER_PG_USERNAME }
      password: { DAGSTER_PG_PASSWORD }
      hostname: { DAGSTER_PG_HOSTNAME }
      db_name: { DAGSTER_PG_DB }
      port: 5432
But in the dagster-postgres library documentation: https://docs.dagster.io/_apidocs/libraries/dagster-postgres it lists each as separate:
event_log_storage:
  module: dagster_postgres.event_log
  class: PostgresEventLogStorage
  config:
    postgres_db:
      username: { username }
      password: { password }
      hostname: { hostname }
      db_name: { db_name }
      port: { port }
What's the difference? Is
storage
config equivalent to adding the three:
event_log_storage
,
run_storage
and
schedule_storage
?
r

rex

10/18/2022, 4:11 PM
Hi Matthew - either would work. We were doing some work to consolidate this configuration, hence why in the main docs, the unified storage configuration is shown. We should probably clarify the API docs though Under the hood, we just interpret it as separate configuration for
event_log_storage
,
run_storage
, and
schedule_storage
. cc @prha
m

Matthew Whiteside

10/18/2022, 4:13 PM
Thanks. Thats what i assumed, but just wanted to confirm.