https://dagster.io/ logo
Title
q

Qumber Ali

11/16/2021, 9:36 AM
Hi all, how I can switch all these logs to Postgres db instead of any directory.
event_log_storage:
  module: dagster.core.storage.event_log
  class: SqliteEventLogStorage
  config:
    base_dir: /home/ubuntu/dagster_home/history/runs/
compute_logs:
  module: dagster.core.storage.local_compute_log_manager
  class: LocalComputeLogManager
  config:
    base_dir: /home/ubuntu/dagster_home/storage
schedule_storage:
  module: dagster.core.storage.schedules
  class: SqliteScheduleStorage
  config:
    base_dir: /home/ubuntu/dagster_home/schedules
I have done for run_storage like this
run_storage:
  module: dagster_postgres.run_storage
  class: PostgresRunStorage
  config:
    postgres_db:
      db_name: dagster
      hostname: 127.0.0.1
      password: password
      port: 5432
      username: postgres
j

johann

11/16/2021, 2:12 PM
This doc covers the options for these storages https://docs.dagster.io/deployment/dagster-instance
q

Qumber Ali

11/16/2021, 2:14 PM
Already done thanks, just tell me one thing if I change any thing in dagster.yaml should I need to restart the dagster-daemon or restart dagit is enough?
j

johann

11/16/2021, 2:14 PM
Yes you should restart both processes
q

Qumber Ali

11/16/2021, 2:15 PM
but when I restart daemon all running and queued job got cancelled is this the actually flow of dagster?
j

johann

11/16/2021, 2:15 PM
(And ensure that both are pointed at the same dagster.yaml, for example by setting DAGSTER_HOME for both to a directory containing it)
q

Qumber Ali

11/16/2021, 2:15 PM
yeah I'm doing that.
j

johann

11/16/2021, 2:16 PM
It sounds like you’re using the default run launcher, which means your runs are taking place on the daemon process. In that case yes, your runs will be terminated when you bring down the process.
Other run launchers (docker, k8s, etc.) execute runs in an isolated process
q

Qumber Ali

11/16/2021, 2:18 PM
So should I use any of them?
j

johann

11/16/2021, 2:25 PM
It’s really up to your needs. In general, running everything on a single host will probably hit scaling limits eventually (even besides this rolling upgrade issue). But it is a bit simpler