Ben Lindsay
07/03/2020, 6:19 PMexecute_pipeline()
or using dagster pipeline execute -f my_pipeline.py
, I'm not seeing those in the Runs tab when I open the GUI with dagit -f my_pipeline.py
. Is it possible to get that working, and if so, can someone help me? Thanks in advance!sashank
07/03/2020, 6:21 PM$DAGSTER_HOME
yet?Ben Lindsay
07/03/2020, 6:22 PMsashank
07/03/2020, 6:24 PMDAGSTER_HOME
to the folders pathdagster.yaml
file to further configure your setupBen Lindsay
07/03/2020, 6:27 PM$DAGSTER_HOME
in docker-based projects? like /usr/src/app/.dagster_home
or something?sashank
07/03/2020, 6:41 PM/opt/dagster/dagster_home
Ben Lindsay
07/03/2020, 6:47 PMexecute_pipeline
to show up in the GUI or to write any data to $DAGSTER_HOME
. As an MVP of this, I have hello_dagster.py
that looks like this:
from dagster import execute_pipeline, pipeline, solid
@solid
def get_name(_):
return "dagster"
@solid
def hello(context, name: str):
<http://context.log.info|context.log.info>("Hello, {name}!".format(name=name))
@pipeline
def hello_pipeline():
hello(get_name())
if __name__ == "__main__":
execute_pipeline(hello_pipeline)
Before I executed the pipelines, I ran
$ export DAGSTER_HOME=$(pwd)/.dagster_home
$ mkdir -p .dagster_home
If I run
$ dagster pipeline execute -f hello_dagster.py
then the pipeline executes, the run shows up in dagit, and I can see entries added to .dagster_home/history/runs
. If I run
$ python hello_dagster.py
then the pipeline executes, but I don't see anything written to .dagster_home
and no new runs show up in dagit. Any ideas why this might be?alex
07/06/2020, 3:07 PMexecute_pipeline
API takes a parameter for the instance
and defaults to using an ephemeral one (since most of the time this API is used for tests.
Just add instance=DagsterInstance.get()
to get the behavior you desireBen Lindsay
07/07/2020, 6:56 PM