https://dagster.io/ logo
Title
h

Herbert Ilhan Tanujaya

02/28/2023, 7:44 AM
[SOLVED] Hi, I’ve been trying to deploy to Dagster Cloud but I got an error I don’t understand. No logs to debug either. Posting the logs as a reply, any help appreciated 🙂
Building project dependencies for Python 3.8, writing to /var/folders/fp/75kvyxcj6td_58znsqwk66rm0000gn/T/tmpds0412oz
Warning: Failed to build dependencies in current environment, falling back to build in a docker environment
Running docker <http://ghcr.io/dagster-io/dagster-manylinux-builder:latest|ghcr.io/dagster-io/dagster-manylinux-builder:latest>
Mapped folders:
 - /var/folders/fp/75kvyxcj6td_58znsqwk66rm0000gn/T/tmpds0412oz -> /output
Error: Failed to build dependencies using docker
Error: ╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /opt/python/cp38-cp38/lib/python3.8/site-packages/dagster_cloud_cli/commands │
│ /serverless/__init__.py:427 in build_python_dependencies                     │
│                                                                              │
│   424 │   pex_flags: str,                                                    │
│   425 ):                                                                     │
│   426 │   ui.print(f"Going to build Python dependencies pex from {requiremen │
│ ❱ 427 │   pex_builder.deps.build_deps_from_requirements_file(                │
│   428 │   │   requirements_path, output_pex_path, json.loads(pex_flags)      │
│   429 │   )                                                                  │
│   430                                                                        │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │   output_pex_path = '/output/deps-from-ef3fb9d507aeb380ebae4d1e6e2d18e1… │ │
│ │         pex_flags = '["--platform=manylinux2014_x86_64-cp-38-cp38",      │ │
│ │                     "--no-strip-pex-env", "--pip-vers'+41                │ │
│ │ requirements_path = '/output/deps-requirements-ef3fb9d507aeb380ebae4d1e… │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /opt/python/cp38-cp38/lib/python3.8/site-packages/dagster_cloud_cli/core/pex │
│ _builder/deps.py:289 in build_deps_from_requirements_file                    │
│                                                                              │
│   286 │   │   │   │   logging.warning(proc.stderr.decode("utf-8"))           │
│   287 │   │   │   │   logging.warning("Will retry building deps with a diffe │
│   288 │   │   │   else:                                                      │
│ ❱ 289 │   │   │   │   raise DepsBuildFailure(proc)                           │
│   290 │   │   else:                                                          │
│   291 │   │   │   break                                                      │
│   292                                                                        │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │           add_on_flags = ['--resolver-version=pip-2020-resolver']        │ │
│ │ deps_requirements_path = '/output/deps-requirements-ef3fb9d507aeb380eba… │ │
│ │        output_pex_path = '/output/deps-from-ef3fb9d507aeb380ebae4d1e6e2… │ │
│ │              pex_flags = [                                               │ │
│ │                          │                                               │ │
│ │                          '--platform=manylinux2014_x86_64-cp-38-cp38',   │ │
│ │                          │   '--no-strip-pex-env',                       │ │
│ │                          │   '--pip-version=22.2.2',                     │ │
│ │                          │   '--resolve-local-platforms',                │ │
│ │                          │   '--resolver-version=pip-2020-resolver'      │ │
│ │                          ]                                               │ │
│ │                   proc = CompletedProcess(args=['/opt/python/cp38-cp38/… │ │
│ │                          '-m', 'pex',                                    │ │
│ │                          '--platform=manylinux2014_x86_64-cp-38-cp38',   │ │
│ │                          '--no-strip-pex-env', '--pip-version=22.2.2',   │ │
│ │                          '--resolve-local-platforms',                    │ │
│ │                          '--resolver-version=pip-2020-resolver', '-r',   │ │
│ │                          '/output/deps-requirements-ef3fb9d507aeb380eba… │ │
│ │                          '-o',                                           │ │
│ │                          '/output/deps-from-ef3fb9d507aeb380ebae4d1e6e2… │ │
│ │                          returncode=1, stdout=b'', stderr=b'pid 284 ->   │ │
│ │                          /root/.pex/venvs/0c9f19ef5fa1cf8671bbf48e92b51… │ │
│ │                          -sE                                             │ │
│ │                          /root/.pex/venvs/0c9f19ef5fa1cf8671bbf48e92b51… │ │
│ │                          --disable-pip-version-check                     │ │
│ │                          --no-python-version-warning --exists-action a   │ │
│ │                          --no-input --isolated -q --cache-dir            │ │
│ │                          /root/.pex/pip_cache --log                      │ │
│ │                          /tmp/pex-pip-log.2z1s6n45/pip.log download      │ │
│ │                          --dest                                          │ │
│ │                          /root/.pex/downloads/resolver_download.u5l0n5p… │ │
│ │                          --requirement                                   │ │
│ │                          /output/deps-requirements-ef3fb9d507aeb380ebae… │ │
│ │                          --index-url <https://pypi.org/simple> --retries 5 │ │
│ │                          --timeout 15 exited with 1 and STDERR:\n        │ │
│ │                          error: subprocess-exited-with-error\n  \n       │ │
│ │                          \xc3\x97 python setup.py egg_info did not run   │ │
│ │                          successfully.\n  \xe2\x94\x82 exit code: 1\n    │ │
│ │                          \xe2\x95\xb0\xe2\x94\x80> See above for         │ │
│ │                          output.\n  \n  note: This error originates from │ │
│ │                          a subprocess, and is likely not a problem with  │ │
│ │                          pip.\nerror:                                    │ │
│ │                          metadata-generation-failed\n\n\xc3\x97          │ │
│ │                          Encountered error while generating package      │ │
│ │                          metadata.\n\xe2\x95\xb0\xe2\x94\x80> See above  │ │
│ │                          for output.\n\nnote: This is an issue with the  │ │
│ │                          package mentioned above, not pip.\nhint: See    │ │
│ │                          above for details.\n\n')                        │ │
│ │              try_flags = []                                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
DepsBuildFailure: CompletedProcess(args=['/opt/python/cp38-cp38/bin/python3.8', 
'-m', 'pex', '--platform=manylinux2014_x86_64-cp-38-cp38', '--no-strip-pex-env',
'--pip-version=22.2.2', '--resolve-local-platforms', 
'--resolver-version=pip-2020-resolver', '-r', 
'/output/deps-requirements-ef3fb9d507aeb380ebae4d1e6e2d18e115d3c4da.txt', '-o', 
'/output/deps-from-ef3fb9d507aeb380ebae4d1e6e2d18e115d3c4da.pex'], returncode=1,
stdout=b'', stderr=b'pid 284 -> 
/root/.pex/venvs/0c9f19ef5fa1cf8671bbf48e92b5182927631202/b8ce33d137022c6ee7d0b3
19035a1c56a7bcaa1e/bin/python -sE 
/root/.pex/venvs/0c9f19ef5fa1cf8671bbf48e92b5182927631202/b8ce33d137022c6ee7d0b3
19035a1c56a7bcaa1e/pex --disable-pip-version-check --no-python-version-warning 
--exists-action a --no-input --isolated -q --cache-dir /root/.pex/pip_cache 
--log /tmp/pex-pip-log.2z1s6n45/pip.log download --dest 
/root/.pex/downloads/resolver_download.u5l0n5ps/opt._internal.cpython-3.8.16.bin
.python3.8 --requirement 
/output/deps-requirements-ef3fb9d507aeb380ebae4d1e6e2d18e115d3c4da.txt 
--index-url <https://pypi.org/simple> --retries 5 --timeout 15 exited with 1 and 
STDERR:\n  error: subprocess-exited-with-error\n  \n  \xc3\x97 python setup.py 
egg_info did not run successfully.\n  \xe2\x94\x82 exit code: 1\n  
\xe2\x95\xb0\xe2\x94\x80> See above for output.\n  \n  note: This error 
originates from a subprocess, and is likely not a problem with pip.\nerror: 
metadata-generation-failed\n\n\xc3\x97 Encountered error while generating 
package metadata.\n\xe2\x95\xb0\xe2\x94\x80> See above for output.\n\nnote: This
is an issue with the package mentioned above, not pip.\nhint: See above for 
details.\n\n')
Nevermind, I got it solved! Using
deploy-docker
allows me to see the error messages clearly. Only after successfuly setting up with it, I switched back to
deploy-python-executable
🙂
d

daniel

02/28/2023, 3:43 PM
Ah great - I'll raise this with the team and see why the error was unclear. Is it possible to share what the problem ended up being?