@Scout CI is failing to deploy on step "Python Executable Deploy", help me fix it. Here are the logs:

Deploying to a full deployment: prod
Running on Ubuntu 20.04
Not in a pull request, will not post PR comment
Running ['generated/gha/dagster-cloud.pex', '-m', 'dagster_cloud_cli.entrypoint', 'serverless', 'deploy-python-executable', '/home/runner/work/mycelium/mycelium/project-repo', '--python-version=3.11', '--deps-cache-to=Unsigned-Research/mycelium/master', '--deps-cache-from=Unsigned-Research/mycelium/master', '--build-method=local', '--location-name=*', '--location-file=/home/runner/work/mycelium/mycelium/project-repo/dagster_cloud.yaml', '--git-url=<https://github.com/Unsigned-Research/mycelium/tree/827f1cc81825948ecc5b0954121d47df9a3fd124>', '--commit-hash=827f1cc81825948ecc5b0954121d47df9a3fd124', '--url=<http://unsigned-research.dagster.cloud/prod>', '--location-load-timeout=3600', '--agent-heartbeat-timeout=90']
Going to deploy location mycelium
Building Python executable for mycelium from directory /home/runner/work/mycelium/mycelium/project-repo/. and Python 3.11.
Building project dependencies for Python 3.11, writing to /tmp/tmpdb97y21u
Error: Failed to build dependencies:
Dependency build failure details:
Command:
| /home/runner/work/_actions/dagster-io/dagster-cloud-action/v0.1/generated/gha/dagster-cloud.pex -m pex --platform=manylinux2014_x86_64-cp-311-cp311 --no-strip-pex-env --pip-version=latest -v -v --resolve-local-platforms --resolver-version=pip-2020-resolver -r /tmp/tmpdb97y21u/deps-requirements-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.txt -o /tmp/tmpdb97y21u/deps-from-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.pex
Error:
| pex: Searching for local interpreters matching manylinux2014_x86_64-cp-311-cp311
| pex: Hashing pex
| pex: Hashing pex: 48.0ms
| pex: Isolating pex
| pex: Isolating pex :: Extracting pex to /home/runner/.pex/isolated/ac60ac27c85fa1751e8e471a569159e88a033b8f
| pex: Isolating pex: 23.8ms
| pex:   Extracting pex to /home/runner/.pex/isolated/ac60ac27c85fa1751e8e471a569159e88a033b8f: 23.5ms
| pex: Resolved PythonInterpreter('/opt/hostedtoolcache/Python/3.11.8/x64/bin/python3.11', PythonIdentity('/opt/hostedtoolcache/Python/3.11.8/x64/bin/python3.11', 'cp311', 'cp311', 'manylinux_2_31_x86_64', (3, 11, 8))) for platform manylinux2014_x86_64-cp-311-cp311
| pex: Searching for local interpreters matching manylinux2014_x86_64-cp-311-cp311: 1711.5ms
| pex: Building pex
| pex: Building pex :: Resolving distributions (/tmp/tmpdb97y21u/deps-requirements-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.txt)
| pex: Building pex :: Resolving distributions (/tmp/tmpdb97y21u/deps-requirements-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.txt) :: Resolving requirements.
| pex: Building pex :: Resolving distributions (/tmp/tmpdb97y21u/deps-requirements-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.txt) :: Resolving requirements. :: Resolving for:
|   /opt/hostedtoolcache/Python/3.11.8/x64/bin/python3.11
| pex: Resolving for:
|   /opt/hostedtoolcache/Python/3.11.8/x64/bin/python3.11
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate.csh
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate.fish
| pex: Searching dependency cache: /home/runner/.pex/pip/20.3.4-patched/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps
| pex: Searching dependency cache: /home/runner/.pex/pip/20.3.4-patched/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding pip 20.3.4
| pex: Searching dependency cache: /home/runner/.pex/pip/20.3.4-patched/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding setuptools 44.0.0+3acb925dd708430aeaf197ea53ac8a752f7c1863
| pex: Searching dependency cache: /home/runner/.pex/pip/20.3.4-patched/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding wheel 0.37.1
| pex: Searching dependency cache: /home/runner/.pex/pip/20.3.4-patched/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps: 2.9ms
| pex:   Adding pip 20.3.4: 0.0ms
| pex:   Adding setuptools 44.0.0+3acb925dd708430aeaf197ea53ac8a752f7c1863: 0.0ms
| pex:   Adding wheel 0.37.1: 0.0ms
| pex: Resolving pip==20.3.4
| pex: Resolving pip==20.3.4: 0.1ms
| pex: Resolving setuptools==44.0.0+3acb925dd708430aeaf197ea53ac8a752f7c1863
| pex: Resolving setuptools==44.0.0+3acb925dd708430aeaf197ea53ac8a752f7c1863: 0.3ms
| pex: Resolving wheel==0.37.1
| pex: Resolving wheel==0.37.1: 0.2ms
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/easy_install
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/easy_install-3.8
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip3
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip3.8
| pex: Re-writing /home/runner/.pex/venvs/ed91772ee6eac99a45d5b18ffca7698191302da4/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/wheel
| pex: Resolving for:
|   /opt/hostedtoolcache/Python/3.11.8/x64/bin/python3.11: 3315.4ms
| pex: Calculating project names for direct requirements:
|   PyPIRequirement(line=LogicalLine(raw_text='pip==23.2', processed_text='pip==23.2', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='pip', url=None, extras=frozenset(), specifier=<SpecifierSet('==23.2')>, marker=None), editable=False)
|   PyPIRequirement(line=LogicalLine(raw_text='setuptools==68.0.0', processed_text='setuptools==68.0.0', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='setuptools', url=None, extras=frozenset(), specifier=<SpecifierSet('==68.0.0')>, marker=None), editable=False)
|   PyPIRequirement(line=LogicalLine(raw_text='wheel==0.40.0', processed_text='wheel==0.40.0', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='wheel', url=None, extras=frozenset(), specifier=<SpecifierSet('==0.40.0')>, marker=None), editable=False)
| pex: Calculating project names for direct requirements:
|   PyPIRequirement(line=LogicalLine(raw_text='pip==23.2', processed_text='pip==23.2', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='pip', url=None, extras=frozenset(), specifier=<SpecifierSet('==23.2')>, marker=None), editable=False)
|   PyPIRequirement(line=LogicalLine(raw_text='setuptools==68.0.0', processed_text='setuptools==68.0.0', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='setuptools', url=None, extras=frozenset(), specifier=<SpecifierSet('==68.0.0')>, marker=None), editable=False)
|   PyPIRequirement(line=LogicalLine(raw_text='wheel==0.40.0', processed_text='wheel==0.40.0', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='wheel', url=None, extras=frozenset(), specifier=<SpecifierSet('==0.40.0')>, marker=None), editable=False): 0.1ms
| pex: Installing 3 distributions
| pex: Installing /home/runner/.pex/downloads/resolver_download.6c7hlt6n/opt.hostedtoolcache.Python.3.11.8.x64.bin.python3.11/setuptools-68.0.0-py3-none-any.whl in /home/runner/.pex/installed_wheels/11e52c67415a381d10d6b462ced9cfb97066179f0e871399e006c4ab101fc85f/setuptools-68.0.0-py3-none-any.whl
| pex: Installing /home/runner/.pex/downloads/resolver_download.6c7hlt6n/opt.hostedtoolcache.Python.3.11.8.x64.bin.python3.11/wheel-0.40.0-py3-none-any.whl in /home/runner/.pex/installed_wheels/d236b20e7cb522daf2390fa84c55eea81c5c30190f90f29ae2ca1ad8355bf247/wheel-0.40.0-py3-none-any.whl
| pex: Installing /home/runner/.pex/downloads/resolver_download.6c7hlt6n/opt.hostedtoolcache.Python.3.11.8.x64.bin.python3.11/pip-23.2-py3-none-any.whl in /home/runner/.pex/installed_wheels/78e5353a9dda374b462f2054f83a7b63f3f065c98236a68361845c1b0ee7e35f/pip-23.2-py3-none-any.whl
| pex: Installing 3 distributions: 1070.2ms
| pex: Checking install
| pex: Checking install: 1.7ms
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate.csh
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/activate.fish
| pex: Searching dependency cache: /home/runner/.pex/pip/23.2/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps
| pex: Searching dependency cache: /home/runner/.pex/pip/23.2/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding pip 23.2
| pex: Searching dependency cache: /home/runner/.pex/pip/23.2/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding setuptools 68.0.0
| pex: Searching dependency cache: /home/runner/.pex/pip/23.2/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps :: Adding wheel 0.40.0
| pex: Searching dependency cache: /home/runner/.pex/pip/23.2/pip.pex/ac60ac27c85fa1751e8e471a569159e88a033b8f/.deps: 9.6ms
| pex:   Adding pip 23.2: 0.0ms
| pex:   Adding setuptools 68.0.0: 0.0ms
| pex:   Adding wheel 0.40.0: 0.0ms
| pex: Resolving pip==23.2
| pex: Resolving pip==23.2: 0.1ms
| pex: Resolving setuptools==68.0.0
| pex: Resolving setuptools==68.0.0: 1.8ms
| pex: Resolving wheel==0.40.0
| pex: Resolving wheel==0.40.0: 0.1ms
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip3
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/pip3.11
| pex: Re-writing /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707.lck.work/bin/wheel
| pid 2140 -> /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707/bin/python -sE /home/runner/.pex/venvs/5131d9dd741d6d195ef4c766ab20afba4a297b9f/09d437a5822e764c0ffc3ea8fe377c2220db2707/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /home/runner/.pex/pip/23.2/pip_cache --log /tmp/pex-pip-log.jid_k0rw/pip.log download --dest /home/runner/.pex/downloads/resolver_download.uk651owh/opt.hostedtoolcache.Python.3.11.8.x64.bin.python3.11 --requirement /tmp/tmpdb97y21u/deps-requirements-d811c9ed98ca4dccaa976fe67b54863ab3ec0b6c.txt --retries 5 --timeout 15 exited with 1 and STDERR:
|   error: subprocess-exited-with-error
|   
|   × python setup.py egg_info did not run successfully.
|   │ exit code: 1
|   ╰─> See above for output.
|   
|   note: This error originates from a subprocess, and is likely not a problem with pip.
| error: metadata-generation-failed
| 
| × Encountered error while generating package metadata.
| ╰─> See above for output.
| 
| note: This is an issue with the package mentioned above, not pip.
| hint: See above for details.
| 

Not in a pull request, will not post PR comment
Error: Failed to deploy Python Executable. Try disabling fast deploys by setting `ENABLE_FAST_DEPLOYS: 'false'` in your .github/workflows/*yml.
Error: Process completed with exit code 1.

The logs indicate that the deployment process is failing during the building of the Python executable, specifically when trying to build the project dependencies for Python 3.11. The error messages suggest that there is a problem with running

python setup.py egg_info

, which is part of the process to install a Python package. Here are a few steps you can take to troubleshoot and potentially resolve the issue: 1. Check Python Version Compatibility: Ensure that all the dependencies in your project are compatible with Python 3.11. If any dependency is not compatible, consider downgrading to a supported Python version or updating the dependencies. 2. Review Dependency Specifications: Look at the

setup.py

file and any

requirements.txt

or

Pipfile

to make sure that the dependencies are specified correctly. Pay attention to any version pins that might be causing conflicts. 3. Disable Fast Deploys: As suggested by the error message at the end of the logs, try disabling fast deploys by setting

ENABLE_FAST_DEPLOYS: 'false'

in your GitHub Actions workflow file. This will change the deployment process and might bypass the issue you're encountering. 4. Check for Subprocess Errors: The logs mention that the error originates from a subprocess, which is likely related to a specific package. Review the logs to identify which package is causing the issue and investigate further. 5. Inspect the Build Environment: Sometimes, issues can arise from the build environment itself. Make sure that the environment where the build is taking place has all the necessary dependencies and configurations. 6. Review the Full Output: The logs provided are truncated and do not show the full output of the error. Review the full output to get more context on what might be going wrong. 7. Consult the Documentation: Check the documentation for any known issues or additional troubleshooting steps related to deploying Python executables in Dagster Cloud. If you continue to face issues after trying these steps, you may want to reach out to the Dagster community or support for further assistance. They might be able to provide more specific guidance based on the details of your deployment and the exact errors you're encountering.

@Scout How can I find more information about why it's failing?

To find more information about why the deployment is failing, you can take the following steps: 1. Examine the Full Logs: The error message you've posted is a snippet. To understand the root cause, you need to look at the complete logs. Often, the actual error is printed before the generic error message. Look for lines that indicate exceptions or errors that occurred before the deployment process was halted. 2. Check Dependency Compatibility: Since the error occurred during the building of project dependencies, check if all the dependencies are compatible with Python 3.11. You can do this by looking at the documentation for each dependency or by searching for compatibility issues reported by other users. 3. Run the Build Locally: Try to replicate the build process on your local machine using the same Python version (3.11). This can sometimes surface issues that are not immediately apparent in CI logs. 4. Verbose Output: If possible, modify the build command to include a verbose flag (often

-v

or

--verbose

) to get more detailed output during the build process. This can provide additional clues about what's going wrong. 5. Check for Known Issues: Look at the issue trackers for your dependencies and for the tools being used in the build process (like

pex

or

pip

). Others may have encountered the same problem, and there might be discussions or solutions available. 6. Consult the Documentation: The documentation may have sections on troubleshooting or known issues with deployment. It might also have guidelines on how to deploy Python executables with specific versions or configurations. 7. Seek Community Help: If you're unable to resolve the issue on your own, consider asking for help from the Dagster community. You can post a question with the relevant logs and details of the steps you've already taken to troubleshoot the issue. Remember to redact any sensitive information from the logs before sharing them publicly or with support teams.