Paul Fentress
06/03/2023, 7:17 PM# My docker image:
FROM python:3.8-slim-buster
# This is to deal with an issue with the psutil package.
RUN apt-get update -y && apt-get install -y gcc
RUN apt-get -y install xauth
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
EXPOSE 3000
CMD dagster dev -f run_dagster_pipeline.py
# Building the container.
docker build -t py-gce .
# Output:
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 553B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for <http://docker.io/docker/dockerfile:1|docker.io/docker/dockerfile:1> 1.2s
=> CACHED <docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a081764> 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load metadata for <http://docker.io/library/python:3.8-slim-buster|docker.io/library/python:3.8-slim-buster> 0.8s
=> [internal] load build context 0.2s
=> => transferring context: 375.92kB 0.2s
=> [1/6] FROM <http://docker.io/library/python:3.8-slim-buster@sha256:eb48d017c5e117d9fbcbe991b4dbc61|docker.io/library/python:3.8-slim-buster@sha256:eb48d017c5e117d9fbcbe991b4dbc61> 0.0s
=> CACHED [2/6] RUN apt-get update -y && apt-get install -y gcc 0.0s
=> [3/6] RUN apt-get -y install xauth 1.5s
=> [4/6] COPY requirements.txt requirements.txt 0.0s
=> [5/6] RUN pip3 install -r requirements.txt 68.0s
=> [6/6] COPY . . 0.3s
=> exporting to image 2.2s
=> => exporting layers 2.2s
=> => writing image sha256:34ffd1197f7235be0493d3be985d70a053c2a48b9c421ee328e0f324b041dd13 0.0s
=> => naming to <http://docker.io/library/py-gce|docker.io/library/py-gce> 0.0s
# Running the container and porting to different port.
docker run -p 8080:3000 py-gce
# Output:
2023-06-03 17:37:12 +0000 - dagster - INFO - Using temporary directory /tmpkcsq9so9 for storage. This will be removed when dagster dev exits.
2023-06-03 17:37:12 +0000 - dagster - INFO - To persist information across sessions, set the environment variable DAGSTER_HOME to a directory to use.
2023-06-03 17:37:12 +0000 - dagster - INFO - Launching Dagster services...
Telemetry:
As an open source project, we collect usage statistics to inform development priorities. For more
information, read <https://docs.dagster.io/getting-started/telemetry>.
We will not see or store solid definitions, pipeline definitions, modes, resources, context, or
any data that is processed within solids and pipelines.
To opt-out, add the following to $DAGSTER_HOME/dagster.yaml, creating that file if necessary:
telemetry:
enabled: false
Welcome to Dagster!
If you have any questions or would like to engage with the Dagster team, please join us on Slack
(<<https://bit.ly/39dvSsF>>).
2023-06-03 17:37:14 +0000 - dagster.daemon - INFO - Instance is configured with the following daemons: ['AssetDaemon', 'BackfillDaemon', 'SchedulerDaemon', 'SensorDaemon']
2023-06-03 17:37:14 +0000 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
2023-06-03 17:37:14 +0000 - dagit - INFO - Serving dagit on <<http://127.0.0.1:3000>> in process 9
2023-06-03 17:38:14 +0000 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
# Go to open the web UI trying these URLs:
<<http://localhost:8080/>>
<<http://localhost:3000/>>
<http://localhost:127>.0.0.1:3000
<http://localhost:127>.0.0.1:8080
<http://localhost:0>.0.0.0:8080
<http://localhost:0>.0.0.0:3000
<<http://127.0.0.1:3000>>
<http://127.0.0.1:0>.0.0.0:3000
<<http://0.0.0.0:3000>>
<<http://127.0.0.1:8080>>
<http://127.0.0.1:0>.0.0.0:8080
<<http://0.0.0.0:8080>>
But all I get is:
"This sight cant be reached"
# The next thing I try is to search the IP address for my docker container
# using the following commands:
docker ps
# To get my docker image ID
docker inspect "my address"
# I search for my DOCKER IP address and get:
"IPAddress": "172.17.0.2",
# Then I try these URL's
<http://localhost:172>.17.0.2:8080
<http://localhost:172>.17.0.2
<http://localhost:172>.17.0.2:3000
<<http://172.17.0.2:8080>>
None of those work; however, this time it fails because it got stuck, and not because it was immediately denied.
I found this post where they state that: “The 172.17.0.2
IP address is internal to the virtual machine that is running the Docker engine. You won’t be able to access it from your Windows host. The actual IP address of that virtual machine should be 192.168.99.100
which is the IP that ports would be mapped to if, in fact, you mapped any ports.”
https://forums.docker.com/t/unable-to-access-my-first-container-on-172-17-0-2/54106
It appears they are obtain the 192.168.99.100
IP from running docker-machine IP; however, I do not have docker machine on my machine. I downloaded it, but still failed, then went onto read that 192.168.99.100
is the default value so should not be unique for my machine anyways.
"This sight cant be reached: 192.168.99.100 took too long to respond"
I am pretty lost at this point, have been trying for hours to resolve this seemingly simple task!
Also, like shown in my first diagram, this is just a step towards my final goal of opening the Dagster UI from inside a docker container inside a GCE. So if you have any tips for that endevor, feel free to make suggestions for that as well.
I have been all up down stack overflow and found similar issues, but the solutions are not working for me. I am not sure if this is a Dagster issue or Docker issue.