Milos Tomic
08/11/2021, 11:17 AMdagster not found
when I try to do multi-stage docker build using python3.7-slim as building image, and google gcr.io/distroless/python3:debug as running imageFROM python:3.7.5-slim AS compile-image
## install dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends gcc
## virtualenv
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN pip install --upgrade pip && pip install pip-tools
COPY ./app/src/requirements.txt .
RUN pip install -r requirements.txt
## build-image
# FROM python:3.7-slim AS runtime-image
FROM <http://gcr.io/distroless/python3:debug|gcr.io/distroless/python3:debug>
SHELL ["/busybox/sh", "-c"]
# ## install dependencies
# RUN apt-get update && \
# apt-get install -y --no-install-recommends gcc
COPY --from=compile-image /opt/venv /opt/venv
# create dagster home and copy dagster instance configuration
RUN mkdir -p /opt/dagster/dagster_home
RUN mkdir -p /opt/dagster/dagster_home/json
COPY ./app/dagster_home/dagster.worker.yaml /opt/dagster/dagster_home/dagster.yaml
COPY ./app/dagster_home/sftp.yaml /opt/dagster/dagster_home/sftp.yaml
# create app folder and move all the app files
RUN mkdir -p /opt/dagster/app
WORKDIR /opt/dagster/app
COPY ./app/src .
COPY ./Dockerfiles/pipeline/entrypoint.sh entrypoint.sh
## set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PATH="/opt/venv/bin:$PATH"
RUN chmod +x entrypoint.sh
EXPOSE 80
ENTRYPOINT ["/opt/dagster/app/entrypoint.sh"]
alex
08/11/2021, 6:45 PMdagster not found
? Do you have a full stack trace?Milos Tomic
08/12/2021, 1:21 PMFROM python:3.7-slim AS builder
RUN mkdir /install
WORKDIR /install
COPY ./app/src/requirements.txt /requirements.txt
RUN pip install --target="/install" -r /requirements.txt
# running image
FROM <http://gcr.io/distroless/python3:debug|gcr.io/distroless/python3:debug>
SHELL ["/busybox/sh", "-c"]
# create dagster home and copy dagster instance configuration
RUN mkdir -p /opt/dagster/dagster_home
RUN mkdir -p /opt/dagster/dagster_home/json
COPY ./app/dagster_home/dagster.worker.yaml /opt/dagster/dagster_home/dagster.yaml
COPY ./app/dagster_home/sftp.yaml /opt/dagster/dagster_home/sftp.yaml
# create app folder and move all the app files
RUN mkdir -p /opt/dagster/app
WORKDIR /opt/dagster/app
COPY ./app/src .
COPY --from=builder /install /usr/lib/python3.7/site-packages
ENV PYTHONPATH=/usr/lib/python3.7/site-packages/
WORKDIR /opt/dagster/app
EXPOSE 80
ENV DAGSTER_HOME=/opt/dagster/dagster_home
CMD ["dagster", "pipeline execute -p worker $PIPELINE"]
/usr/bin/python3.7: can't open file 'dagster': [Errno 2] No such file or directory
alex
08/12/2021, 2:39 PMdagster
via you requirements.txt ?