For example, I have an entrypoint on my

Dockerfile

that sources a virtualenv and then runs the dagster api, but what I see in the pod description is

Args:
      /opt/maze-env/bin/python
      -m
      dagster
      api
      execute_run
      {"__class__": "..."}

It seems that the first argument is being constructed using

sys.executable

Are there any approaches that would allow us to activate a virtual environment in the job pod?

for context: https://dagster.slack.com/archives/C01U954MEER/p1635257521137700?thread_ts=1634638764.338800&cid=C01U954MEER

cc @daniel can you specify the executable path in the helm chart?

The issue isn't the executable path, that is being picked up fine.

what you seemed to have before was just:

command = ["dagster", "api", "execute_run", input_json]

And that would have actually worked for us. The replacement of:

command = args.get_command_args()

is what broke things for us. Admittedly it was an ugly solution on our end. Using

sys.executable

with

-m

and

dagster

is probably the correct way to go about this for the launcher

And sorry, I believe I'm not following what was going on in that second link, where docker-compose is being referenced. If we could override that somehow we could solve this

So, looking at: https://docs.dagster.io/_apidocs/cli#cmdoption-dagster-api-grpc-use-python-environment-entry-point

so, "If this flag is set, the server will signal to clients that they should launch dagster commands using <this server’s python executable> -m dagster, instead of the default dagster entry point. "

So I'm surprised to see that your arguments are

/opt/maze-env/bin/python
      -m
      dagster

"When running your own gRPC server to serve Dagster code, jobs that launch in a container using code from that server will now default to using dagster as the entry point. Previously, the jobs would run using PYTHON_EXECUTABLE -m dagster, where PYTHON_EXECUTABLE was the value of sys.executable on the gRPC server. For the vast majority of Dagster jobs, these entry points will be equivalent. To keep the old behavior (for example, if you have multiple Python virtualenvs in your image and want to ensure that runs also launch in a certain virtualenv), you can launch the gRPC server using the new ----use-python-environment-entry-point command-line arg."

that's why I asked what version you're on earlier - are you sure your code is past that version?

I'll double-check that it still works this way though

Oh jeez! It seems the dagster daemon is on 13.9!

heh, btw, this comment: https://github.com/dagster-io/dagster/pull/5415#pullrequestreview-791128540 is exactly our issue

I really appreciate y'all taking the time to respond to this, btw.

and our approach has been to wrap entrypoints with a script that first activates the environment. Hacky for sure.

how did you upgrade your daemon?

if it was through the helm chart, the workspace.yaml should be mounted as a volume and the dagster_home would have already been set to find the workspace.

what helm chart version are you running as well?

Thanks again for the help!