The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi,
I'm trying to understand how to manage the `dagster.yaml` configuration.
From the <https://docs.dagster.io/docs/apidocs/internals#instance|instance documentation> I understand that it is read from the `$DAGSTER_HOME` environment.
If the `dagster.yaml` has the logic configuration per project  while the `$DAGSTER_HOME` env is  per user, what is the correct manner to use the `dagster.yaml`?
Where should it be saved and how could it be run per project?

this doc is better <https://docs.dagster.io/docs/deploying/instance>

`$DAGSTER_HOME` points at a directory that can contain a `dagster.yaml` to configure your “instance” which is akin to deployment or install

Thx for your quick response.
As a newbee - i'm probably missing something out
In <https://docs.dagster.io/docs/deploying/instance|the link> it is says:
&gt;  Dagster checks whether the environment variable `DAGSTER_HOME` is set
Isn't this "environment variable"  set per user? then it will always point to *one* directory with *one* `dagster.yaml` file.
However each project that I use will want to run a different `dagster.yaml` file,   how do I configure that?

&gt; However each project that I use will want to run a different `dagster.yaml` file
Out of curiosity why is this? Can you tell me more about your set up?

If you want to change your storage per project - you will have to set up multiple folders with `dagster.yaml` and change your `$DAGSTER_HOME` based on which project you are currently working on.

Our tools are currently set up to handle different pipelines/repositories using a shared storage, so I would consider using one instance for working on your projects, at least locally.

Since each project is from a different domain, the logs would be saved in different databases. We will consider your suggestion.

ya this is a workflow style we haven’t optimized for yet - I would encourage you to file an issue describing what you think an ideal workflow would be for multi-instance work

Just submitted an issue:
<https://github.com/dagster-io/dagster/issues/2444>

I set my `DAGSTER_HOME` environment to a folder on my dagster machine.
I set the dagster.yaml configuration to be exactly <https://docs.dagster.io/docs/apidocs/internals#dagster.core.storage.event_log.SqliteEventLogStorage|as in the documentation>  (just updating the base_dir).
I set the permissions of the folder to allow everyone to write to it.
When running the dagster - the job completes without any errors but I don't see any sqlite file.
Do I need to generate a sqlite file or will be generated automatically?
I'm I missing something?

hmm - I wonder if this is just an issue with `base_dir` - do you see the sqlite file in your `DAGSTER_HOME` directory instead?