https://dagster.io/ logo
Title
m

Milos Tomic

08/11/2021, 11:17 AM
Hey, I’m getting error
dagster not found
when I try to do multi-stage docker build using python3.7-slim as building image, and google gcr.io/distroless/python3:debug as running image
FROM python:3.7.5-slim AS compile-image

## install dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc

## virtualenv
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

RUN pip install --upgrade pip && pip install pip-tools
COPY ./app/src/requirements.txt .
RUN pip install -r requirements.txt

## build-image
# FROM python:3.7-slim AS runtime-image 
FROM <http://gcr.io/distroless/python3:debug|gcr.io/distroless/python3:debug>
SHELL ["/busybox/sh", "-c"]
# ## install dependencies
# RUN apt-get update && \
#     apt-get install -y --no-install-recommends gcc

COPY --from=compile-image /opt/venv /opt/venv

# create dagster home and copy dagster instance configuration
RUN mkdir -p /opt/dagster/dagster_home 
RUN mkdir -p /opt/dagster/dagster_home/json

COPY ./app/dagster_home/dagster.worker.yaml /opt/dagster/dagster_home/dagster.yaml
COPY ./app/dagster_home/sftp.yaml /opt/dagster/dagster_home/sftp.yaml

# create app folder and move all the app files
RUN mkdir -p /opt/dagster/app
WORKDIR /opt/dagster/app

COPY ./app/src .
COPY ./Dockerfiles/pipeline/entrypoint.sh entrypoint.sh

## set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PATH="/opt/venv/bin:$PATH"

RUN chmod +x entrypoint.sh

EXPOSE 80

ENTRYPOINT ["/opt/dagster/app/entrypoint.sh"]
a

alex

08/11/2021, 6:45 PM
what is the context of the
dagster not found
? Do you have a full stack trace?
m

Milos Tomic

08/12/2021, 1:21 PM
FROM python:3.7-slim AS builder
RUN mkdir /install
WORKDIR /install
COPY ./app/src/requirements.txt /requirements.txt
RUN pip install --target="/install" -r /requirements.txt

# running image
FROM <http://gcr.io/distroless/python3:debug|gcr.io/distroless/python3:debug>
SHELL ["/busybox/sh", "-c"]

# create dagster home and copy dagster instance configuration
RUN mkdir -p /opt/dagster/dagster_home 
RUN mkdir -p /opt/dagster/dagster_home/json

COPY ./app/dagster_home/dagster.worker.yaml /opt/dagster/dagster_home/dagster.yaml
COPY ./app/dagster_home/sftp.yaml /opt/dagster/dagster_home/sftp.yaml

# create app folder and move all the app files
RUN mkdir -p /opt/dagster/app
WORKDIR /opt/dagster/app

COPY ./app/src .
COPY --from=builder /install /usr/lib/python3.7/site-packages

ENV PYTHONPATH=/usr/lib/python3.7/site-packages/
WORKDIR /opt/dagster/app

EXPOSE 80
ENV DAGSTER_HOME=/opt/dagster/dagster_home
CMD ["dagster", "pipeline execute -p worker $PIPELINE"]
/usr/bin/python3.7: can't open file 'dagster': [Errno 2] No such file or directory
That’s full error
I’m trying to build image using distroless, not sure if dagster is avaiable there
a

alex

08/12/2021, 2:39 PM
are you installing
dagster
via you requirements.txt ?