The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi all, I'm just starting to play with dagster, really like it so far. Question for anyone who can help: I really like the Dagit UI, but I can't figure out a way to view runs that are executed outside the GUI. Like if I run a pipeline using `execute_pipeline()` or using `dagster pipeline execute -f my_pipeline.py`, I'm not seeing those in the Runs tab when I open the GUI with `dagit -f my_pipeline.py`. Is it possible to get that working, and if so, can someone help me? Thanks in advance!

Hey Ben! Have you set up a `$DAGSTER_HOME` yet?

nope! i'll look for that in the docs. thanks!

All you need to do is create a directory and set the env var `DAGSTER_HOME` to the folders path 

<https://docs.dagster.io/overview/instances/dagster-instance|https://docs.dagster.io/overview/instances/dagster-instance>

You can also create a `dagster.yaml` file to further configure your setup  

yep, that worked, thanks for the fast response <@UL7C8DRKN>!

is there a "standard" choice of location for `$DAGSTER_HOME` in docker-based projects? like `/usr/src/app/.dagster_home` or something?

Not really, but you can copy the setup here: <https://docs.dagster.io/deploying/local|https://docs.dagster.io/deploying/local>

This guide uses `/opt/dagster/dagster_home`

<@UL7C8DRKN> Using the CLI and GUI are working fine for me, but for some reason I'm not able to get runs executed with `execute_pipeline` to show up in the GUI or to write any data to `$DAGSTER_HOME` . As an MVP of this, I have `hello_dagster.py` that looks like this:
```from dagster import execute_pipeline, pipeline, solid


@solid
def get_name(_):
    return "dagster"


@solid
def hello(context, name: str):
    <http://context.log.info|context.log.info>("Hello, {name}!".format(name=name))


@pipeline
def hello_pipeline():
    hello(get_name())


if __name__ == "__main__":
    execute_pipeline(hello_pipeline)```
Before I executed the pipelines, I ran
```$ export DAGSTER_HOME=$(pwd)/.dagster_home
$ mkdir -p .dagster_home```
If I run
```$ dagster pipeline execute -f hello_dagster.py```
then the pipeline executes, the run shows up in dagit, and I can see entries added to `.dagster_home/history/runs`. If I run
```$ python hello_dagster.py```
then the pipeline executes, but I don't see anything written to `.dagster_home` and no new runs show up in dagit. Any ideas why this might be?

the `execute_pipeline` API takes a parameter for the `instance` and defaults to using an ephemeral one (since most of the time this API is used for tests.

Just add `instance=DagsterInstance.get()` to get the behavior you desire

Thanks, I'll give this a shot when I get a chance!