https://dagster.io/ logo
#announcements
Title
# announcements
s

Sergii Ivakhno

09/30/2020, 4:25 PM
Hello All! Quick question. I set up
dagster.yml
with
PostgresRunStorage
as
run_storage:
module: dagster_postgres.run_storage
class: PostgresRunStorage
config:
postgres_db:
username:
env: DAGSTER_PG_USERNAME
password:
env: DAGSTER_PG_PASSWORD
hostname:
env: DAGSTER_PG_HOST
db_name:
env: DAGSTER_PG_DB
port: 5432
and similar for events using
PostgresEventLogStorage
. I then execute pipeline with dask executor - it runs successfully but gives quite a few errors
qlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 123145510014976 and this is thread id 123145494249472.
I am curious why
SQLite
is mentioned at all since I use
Postgres
for run storage? Thanks for clarifications!
s

sashank

09/30/2020, 4:26 PM
Hey @sephi, you also need to configure
event_log_storage
to use postgres
s

Sergii Ivakhno

09/30/2020, 4:28 PM
thanks for speedy reply @sashank. That's why I am asking as I used that doc as a reference 🙂. Below is what I have for event_log but still get those errors.
event_log_storage:
module: dagster_postgres.event_log
class: PostgresEventLogStorage
config:
postgres_db:
username:
env: DAGSTER_PG_USERNAME
password:
env: DAGSTER_PG_PASSWORD
hostname:
env: DAGSTER_PG_HOST
db_name:
env: DAGSTER_PG_DB
port: 5432
s

sashank

09/30/2020, 4:29 PM
are you using the scheduler?
there is one more for
schedule_storage
if you are
s

Sergii Ivakhno

09/30/2020, 4:30 PM
No it's not in the config and I don't use it. I did start with SQLite store though and then changed to Postgres. I wander if it's due to local cashing from previous set-up but not sure how to refresh.
s

sashank

09/30/2020, 4:31 PM
are you running these pipelines from dagit or the CLI? just restarting dagit should do the trick
also if you click the “instance details” button in dagit, you can confirm that you’re using postgres
🙌 1
another thing to check: do you have
$DAGSTER_HOME
set, and
dagster.yaml
defined in the folder
$DAGSTER_HOME
points to?
🙌 1
s

Sergii Ivakhno

09/30/2020, 4:50 PM
I checked instance details and can see that schedule_storage was set by default to`SqliteScheduleStorage` as I didn't specify it. Modifying this and restarting dagit solved the issue!