https://dagster.io/ logo
Powered by
k

Kevin

09/23/2020, 5:21 PM
I've got a doozy of a question: running 
dagit -f dagstermill_nb_pipeline.py
runs beautifully; it imports my other .py files and no problems all the pipelines from the repository run. I take this same file and use it with my k8s deployment with grpc and the dagit pod always fails complaining that my .py files weren't found as modules... I've attached the k8s log associated to the failed dagit pod
dagit.log
m

max

09/23/2020, 5:26 PM
are you sure that all the files you want are present on the dagit pod, and that they have the same relative paths?
k

Kevin

09/23/2020, 5:28 PM
98% confident as i pushed a new docker image for k8s to pull, and the new image has all the files
m

max

09/23/2020, 5:32 PM
is your local instance using grpc also
and if not can you enable it
k

Kevin

09/23/2020, 5:33 PM
local instance is not using grpc, i'm just running the dagit command -- how would i go about enabling grpc for a local instance?
m

max

09/23/2020, 5:35 PM
is the environment variable 
DAGSTER_HOME
set?
if not, can you set it to point at some directory
and make sure you have a 
dagster.yaml
file in that directory with the following lines in it:
Copy code
opt_in:
  local_servers: true
or add them to the existing 
dagster.yaml
k

Kevin

09/23/2020, 5:45 PM
still going smoothly with local, after adding your lines to a dagster.yaml
anything you want from the local to look at?
the command i ran was the same: 
dagit -f dagstermill_nb_pipeline.py
c

cat

09/23/2020, 5:50 PM
could you share what you used in the 
dagsterApiGrpcArgs
section of the helm values yaml?
k

Kevin

09/23/2020, 5:52 PM
dagsterApiGrpcArgs: - "-f" - "dagstermill_nb_pipeline.py" - "-a" - "gRPC_rep"
Copy code
@repository
def gRPC_rep():
    return [LDA_NB_pipeline,
            csp.parallel_simple_pipeline,
            cp.parallel_pipeline,
            p.math,
            ]
where csp, cp, and p, are imports from other local .py files
a

alex

09/23/2020, 5:55 PM
I think you need to set the working directory explicitly 
'-d' , './'
🎉 1
we had some gnarly implicit working directory bugs and overcorrected to disallowing working directory based imports unless its explicitly set
we’ve brought the defaulting back on the user facing cli’s, but not yet the programatic cli entries
k

Kevin

09/23/2020, 6:00 PM
alex that is beautiful -- that's just what it needed! thank you all for your help!
d

daniel

09/23/2020, 7:19 PM
Just put out a fix to make that the default behavior for grpc (much like dagit)
😻 1
Powered by