https://dagster.io/ logo
#ask-ai
Title
# ask-ai
d

denton

02/06/2024, 4:32 PM
@Scout in my
dagster.yaml
file, I'd like to save the
event_log_storage
compontents into a separate postgres schema named
dagster-internal
. How would i do that?
s

Scout

02/06/2024, 4:32 PM
To configure your
event_log_storage
to use a specific schema in a Postgres database, you can set the
search_path
to your desired schema in the
dagster.yaml
configuration file. This is done by adding the
params
block under the Postgres configuration and specifying the
options
with the
-c search_path=<schema_name>
setting. Here's an example of how you might configure the
event_log_storage
in your
dagster.yaml
to use a schema named `dagster-internal`:
Copy code
yaml event_log_storage:   module: dagster_postgres.event_log   class: PostgresEventLogStorage   config:     postgres_db:       username: { username }       password: { password }       hostname: { hostname }       db_name: { db_name }       port: { port }     params:       options: -c search_path=dagster-internal
In this configuration, replace
{ username }
,
{ password }
,
{ hostname }
,
{ db_name }
, and
{ port }
with your actual Postgres database credentials. The
search_path
is set to
dagster-internal
, which instructs Postgres to use the
dagster-internal
schema for the event log storage. Please ensure that the schema
dagster-internal
exists in your Postgres database, as Dagster will not create the schema for you. You may need to create it manually if it does not already exist.
d

denton

02/06/2024, 6:05 PM
@Scout i received this error when configuring the yaml like the above. Please advise
Copy code
docker_example_daemon     |     Error 1: Received unexpected config entry "params" at the root. Expected: "{ postgres_db?: { db_name: (String | { env: String }) hostname: (String | { env: String }) params?: { } password: (String | { env: String }) port?: (Int | { env: String }) scheme?: (String | { env: String }) username: (String | { env: String }) } postgres_url?: (String | { env: String }) should_autocreate_tables?: Bool }".
s

Scout

02/06/2024, 6:05 PM
The error message you're seeing indicates that the
params
entry is unexpected in the location where you've placed it in the
dagster.yaml
configuration file. According to the error message, the expected structure does not have
params
at the root level of the
postgres_db
configuration. The documentation does not explicitly mention the use of
params
to set the
search_path
for a Postgres schema. Therefore, the approach I previously suggested may not be supported directly by Dagster's configuration system. Given this, it seems that setting the
search_path
via the
dagster.yaml
is not the correct approach. Instead, you might need to ensure that the
dagster-internal
schema is set as the default for the user or the connection in the Postgres database itself. To set the schema search path for a Postgres user, you can execute the following SQL command in your Postgres database:
Copy code
sql ALTER USER your_username SET search_path TO dagster-internal;
Replace
your_username
with the actual username that Dagster uses to connect to the database. This command will set the default schema search path for the specified user to
dagster-internal
. After setting the search path at the database level, you can continue to use the standard configuration for
event_log_storage
in your
dagster.yaml
without specifying `params`:
Copy code
yaml event_log_storage:   module: dagster_postgres.event_log   class: PostgresEventLogStorage   config:     postgres_db:       username: { username }       password: { password }       hostname: { hostname }       db_name: { db_name }       port: { port }
Make sure to replace the placeholders with your actual database credentials. If you continue to encounter issues or if the above solution does not work for your use case, I recommend checking the official Dagster documentation or reaching out to the Dagster community for further assistance, as there may be updates or specific configurations that are not covered in the current documentation.
2 Views