https://dagster.io/ logo
Powered by
#dagster-serverless
Title
# dagster-serverless
a

Aliénor Lougerstay

01/18/2023, 8:09 PM
Hello! We are using github actions to deploy and want to upgrade from python 3.8 to 3.10 but with the deploy.yml that was provided, the only option to select Python version seems to be with the fast deploy option, which does not work in our case (some issue with PEX). Any other way to select python version correctly, ideally by using slim-3.10 instead of slim-3.8?
d

daniel

01/18/2023, 8:11 PM
Hi Aliénor, slim-3.10 is exactly the way to do this - there are instructions for customizing the base image for docker builds under the "Change the base image" tab here: https://docs.dagster.io/dagster-cloud/deployment/serverless#disabling-pex-based-deploys
We'd be happy to look into the issue with PEX too if that would be helpful
a

Aliénor Lougerstay

01/18/2023, 9:12 PM
That’d be great, here or dm?
d

daniel

01/18/2023, 9:13 PM
If here is an option / not sensitive, that would make it easiest to loop in the PEX experts on the team
👍 1
a

Aliénor Lougerstay

01/18/2023, 9:24 PM
let me try to get back to the issue we were having
Currently stuck in infinite loop of dependency hell 🙂
d

daniel

01/18/2023, 10:16 PM
Ah sorry to hear that 😕 if you're able to pass along logs from the github action (here or via DM) we could take a look
Knowing the setup.py that you're using would be helpful as well if thats' possible
a

Aliénor Lougerstay

01/18/2023, 10:19 PM
setup.py:
Copy code
from setuptools import find_packages, setup

setup(
    name="dagster_project",
    packages=find_packages(exclude=["dagster_project_tests"]),
    install_requires=[
        # Dagster
        "dagster==1.1.9",  # <https://pypi.org/project/dagster/>
        "dagster-cloud==1.1.9",  # <https://pypi.org/project/dagster-cloud/>
        "dagster-aws",  # <https://pypi.org/project/dagster-aws/>
        # dbt
        "dbt-core==1.3.2",
        "dagster-dbt",
        # sql / snowflake
        "pyodbc",  # <https://pypi.org/project/pyodbc/>
        "SQLAlchemy",  # <https://pypi.org/project/SQLAlchemy/>
        "snowflake-sqlalchemy",  # <https://pypi.org/project/snowflake-sqlalchemy/>
        "minimal-snowplow-tracker==0.0.2",
        "snowflake-connector-python",  # <https://pypi.org/project/snowflake-connector-python/>
        # optional dependency of "snowflake-connector-python"
        "fastparquet",  # <https://pypi.org/project/fastparquet/>
        # data
        "numpy",  # <https://pypi.org/project/numpy/>
        "pandas",  # <https://pypi.org/project/pandas/>
        # requests / oauth
        "requests",  # <https://pypi.org/project/requests/>
        "oauthlib",  # <https://pypi.org/project/oauthlib/>
        "requests-oauthlib",  # <https://pypi.org/project/requests-oauthlib/>
        # misc
        "tqdm",  # <https://pypi.org/project/tqdm/>
        "retry",  # <https://pypi.org/project/retry/>
        "Pebble",  # <https://pypi.org/project/Pebble/>
        "StrEnum",  # <https://pypi.org/project/StrEnum/>
        "stackprinter",  # <https://pypi.org/project/stackprinter/>
    ],
    extras_require={"dev": ["dagit", "pytest"]},
)
^ tried adding some versions in hope to resolve conflicts to no avail
d

daniel

01/18/2023, 10:21 PM
are you able to pip install dagster_project locally?
a

Aliénor Lougerstay

01/18/2023, 10:22 PM
Github actions log:
Copy code
ERROR: Cannot install dagster-dbt because these package versions have conflicting dependencies.
  ERROR: ResolutionImpossible: for help visit <https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts>

   The conflict is caused by:
       dbt-core 1.3.2 depends on minimal-snowplow-tracker==0.0.2
       dbt-core 1.3.1 depends on minimal-snowplow-tracker==0.0.2
       dbt-core 1.3.0 depends on minimal-snowplow-tracker==0.0.2

   To fix this you could try to:
   1. loosen the range of package versions you've specified
   2. remove package versions to allow pip attempt to solve the dependency conflict


  WARNING:root:Will retry building deps with a different resolution mechanism
  INFO:root:Running pex with '--python=python3.10 --platform=manylinux2014_x86_64-cp-310-cp310 --no-strip-pex-env --pip-version=22.2.2'
  INFO:root:Running ['/home/runner/work/_actions/dagster-io/dagster-cloud-action/pex-v0.1/generated/gha/builder.pex', '-m', 'pex', '--python=python3.10', '--platform=manylinux2014_x86_64-cp-310-cp310', '--no-strip-pex-env', '--pip-version=22.2.2', '-r', '/home/runner/work/***/***/build/deps-requirements-6cf0130bc3280c83746d47e8b8d7b4716c2139d6.txt', '-o', '/home/runner/work/***/***/build/deps-from-6cf0130bc3280c83746d47e8b8d7b4716c2139d6.pex', '--pex-root', '/home/runner/work/***/***/build/.pex'] in '/home/runner/work/_actions/dagster-io/dagster-cloud-action/pex-v0.1'
  ERROR:root:Failed to build deps.pex
  ERROR:root:
  ERROR:root:pid 1954 -> /home/runner/work/***/***/build/.pex/venvs/86d739740a60f4eb5cdc8b340cefc313bec7343f/0c9ffbfa1b4080a12413e6319eb7d3da7eff4f40/bin/python -sE /home/runner/work/***/***/build/.pex/venvs/86d739740a60f4eb5cdc8b340cefc313bec7343f/0c9ffbfa1b4080a12413e6319eb7d3da7eff4f40/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /home/runner/work/***/***/build/.pex/pip_cache --log /tmp/pex-pip-log.t37ff3h1/pip.log download --dest /home/runner/work/***/***/build/.pex/downloads/resolver_download.h89_rju4/cp310-cp310-manylinux2014_x86_64 --only-binary :all: --requirement /home/runner/work/***/***/build/deps-requirements-6cf0130bc3280c83746d47e8b8d7b4716c2139d6.txt --platform manylinux2014_x86_64 --implementation cp --python-version 310 --abi cp310 --index-url <https://pypi.org/simple> --retries 5 --timeout 15 exited with 1 and STDERR:
We are developing in codespaces using the same setup.py and never had an issue
And yes 
pip install dagster_project
works although does nothing because already installed 🙂
d

daniel

01/18/2023, 10:25 PM
It looks like it's not happy with that
Copy code
"minimal-snowplow-tracker==0.0.2"
line
er its not totally clear
a

Aliénor Lougerstay

01/18/2023, 10:25 PM
Added that line to try to help it, used to not have it at all with same results
d

daniel

01/18/2023, 10:25 PM
i'll see if I can reproduce with that setup.py
❤️ 1
a

Aliénor Lougerstay

01/18/2023, 10:26 PM
and it’s just serverless deployments that do this
to note: we never had any success with PEX, and a very similar setup.py used to work well with slow deployment
s

Shalabh Chaturvedi

01/18/2023, 10:43 PM
Hi Aliénor I looked at the pex log above and have a question. Can you tell me what version of ubuntu is specified in 
runs-on:
 in your 
.github/workflow/deploy.yml
? If it is 
ubuntu-latest
can you try changing it to 
ubuntu-20.04
please?
a

Aliénor Lougerstay

01/18/2023, 10:44 PM
it is latest, will switch
s

Shalabh Chaturvedi

01/18/2023, 10:44 PM
Also do you have the rest of the log for the above log? I expect it handle the failure and fallback to a different build method (using a docker builder image).
a

Aliénor Lougerstay

01/18/2023, 10:49 PM
Rest of the log for the Building PEX Files section:
Copy code
WARNING:root:Will retry building deps with a different resolution mechanism
  INFO:root:Running pex with '--python=python3.10 --platform=manylinux2014_x86_64-cp-310-cp310 --no-strip-pex-env --pip-version=22.2.2'
  INFO:root:Running ['/home/runner/work/_actions/dagster-io/dagster-cloud-action/pex-v0.1/generated/gha/builder.pex', '-m', 'pex', '--python=python3.10', '--platform=manylinux2014_x86_64-cp-310-cp310', '--no-strip-pex-env', '--pip-version=22.2.2', '-r', '/home/runner/work/***/***/build/deps-requirements-57069588bb965f0e6c8f4df3b2175cb5e3641e67.txt', '-o', '/home/runner/work/***/***/build/deps-from-57069588bb965f0e6c8f4df3b2175cb5e3641e67.pex', '--pex-root', '/home/runner/work/***/***/build/.pex'] in '/home/runner/work/_actions/dagster-io/dagster-cloud-action/pex-v0.1'
  ERROR:root:Failed to build deps.pex
  ERROR:root:
  ERROR:root:pid 1883 -> /home/runner/work/***/***/build/.pex/venvs/86d739740a60f4eb5cdc8b340cefc313bec7343f/0c9ffbfa1b4080a12413e6319eb7d3da7eff4f40/bin/python -sE /home/runner/work/***/***/build/.pex/venvs/86d739740a60f4eb5cdc8b340cefc313bec7343f/0c9ffbfa1b4080a12413e6319eb7d3da7eff4f40/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /home/runner/work/***/***/build/.pex/pip_cache --log /tmp/pex-pip-log.8ldce_k7/pip.log download --dest /home/runner/work/***/***/build/.pex/downloads/resolver_download.u38ihtv_/cp310-cp310-manylinux2014_x86_64 --only-binary :all: --requirement /home/runner/work/***/***/build/deps-requirements-57069588bb965f0e6c8f4df3b2175cb5e3641e67.txt --platform manylinux2014_x86_64 --implementation cp --python-version 310 --abi cp310 --index-url <https://pypi.org/simple> --retries 5 --timeout 15 exited with 1 and STDERR:
  Ignoring the following environment variables in Pex venv mode:
  _PEX_FILE_LOCK_STYLE=bsd

  ERROR: Could not find a version that satisfies the requirement logbook<1.6,>=1.5 (from dbt-core~=1.3.0->dbt-snowflake==1.3.0->-r /home/runner/work/***/***/build/deps-requirements-57069588bb965f0e6c8f4df3b2175cb5e3641e67.txt (line 7)) (from versions: none)
  ERROR: No matching distribution found for logbook<1.6,>=1.5 (from dbt-core~=1.3.0->dbt-snowflake==1.3.0->-r /home/runner/work/***/***/build/deps-requirements-57069588bb965f0e6c8f4df3b2175cb5e3641e67.txt (line 7))
Changing Ubuntu version seems to have fixed the PEX file issue!
ubuntu-latest comes from the quickstart deploy.yml file, might be good to update there if that is what causes issues
s

Shalabh Chaturvedi

01/18/2023, 10:55 PM
That's great! Please change the 
runs-on:
in your 
branch_deployments.yml
as well in case you want to branch deployments in serverless. This was a known issue that we recently fixed by pinning our yml files to 
ubuntu-20.04
. However your file was probably deployed before this fix. We also fixed the issue with a fallback for 
ubuntu-latest
for existing users, which is why I expected it to still work, but I will look more into why it failed.
18 Views
Powered by