https://dagster.io/ logo
#announcements
Title
# announcements
b

Ben Lindsay

07/03/2020, 6:19 PM
Hi all, I'm just starting to play with dagster, really like it so far. Question for anyone who can help: I really like the Dagit UI, but I can't figure out a way to view runs that are executed outside the GUI. Like if I run a pipeline using
execute_pipeline()
or using
dagster pipeline execute -f my_pipeline.py
, I'm not seeing those in the Runs tab when I open the GUI with
dagit -f my_pipeline.py
. Is it possible to get that working, and if so, can someone help me? Thanks in advance!
s

sashank

07/03/2020, 6:21 PM
Hey Ben! Have you set up a
$DAGSTER_HOME
yet?
b

Ben Lindsay

07/03/2020, 6:22 PM
nope! i'll look for that in the docs. thanks!
s

sashank

07/03/2020, 6:24 PM
All you need to do is create a directory and set the env var
DAGSTER_HOME
to the folders path
You can also create a
dagster.yaml
file to further configure your setup
b

Ben Lindsay

07/03/2020, 6:27 PM
yep, that worked, thanks for the fast response @sashank!
🎉 1
is there a "standard" choice of location for
$DAGSTER_HOME
in docker-based projects? like
/usr/src/app/.dagster_home
or something?
s

sashank

07/03/2020, 6:41 PM
Not really, but you can copy the setup here: https://docs.dagster.io/deploying/local
This guide uses
/opt/dagster/dagster_home
b

Ben Lindsay

07/03/2020, 6:47 PM
cool, thanks!
@sashank Using the CLI and GUI are working fine for me, but for some reason I'm not able to get runs executed with
execute_pipeline
to show up in the GUI or to write any data to
$DAGSTER_HOME
. As an MVP of this, I have
hello_dagster.py
that looks like this:
Copy code
from dagster import execute_pipeline, pipeline, solid


@solid
def get_name(_):
    return "dagster"


@solid
def hello(context, name: str):
    <http://context.log.info|context.log.info>("Hello, {name}!".format(name=name))


@pipeline
def hello_pipeline():
    hello(get_name())


if __name__ == "__main__":
    execute_pipeline(hello_pipeline)
Before I executed the pipelines, I ran
Copy code
$ export DAGSTER_HOME=$(pwd)/.dagster_home
$ mkdir -p .dagster_home
If I run
Copy code
$ dagster pipeline execute -f hello_dagster.py
then the pipeline executes, the run shows up in dagit, and I can see entries added to
.dagster_home/history/runs
. If I run
Copy code
$ python hello_dagster.py
then the pipeline executes, but I don't see anything written to
.dagster_home
and no new runs show up in dagit. Any ideas why this might be?
a

alex

07/06/2020, 3:07 PM
the
execute_pipeline
API takes a parameter for the
instance
and defaults to using an ephemeral one (since most of the time this API is used for tests. Just add
instance=DagsterInstance.get()
to get the behavior you desire
👍 2
b

Ben Lindsay

07/07/2020, 6:56 PM
Thanks, I'll give this a shot when I get a chance!