Hi, I am trying to use the baked in GitHub actions to deploy to Dagster Serverless, however we are getting the above error in the

Build and deploy to Dagster Cloud serverless

step. See log output below. This is similar to this thread, however there doesn’t appear to be a resolution on that thread. Can someone please assist?

#10 68.47 Collecting dbt-core
#10 68.50   Downloading dbt-core-0.14.4.tar.gz (540 kB)
#10 68.51      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 540.3/540.3 kB 53.2 MB/s eta 0:00:00
#10 68.59   Preparing metadata (setup.py): started
#10 68.90   Preparing metadata (setup.py): finished with status 'done'
#10 68.93 Collecting dbt-postgres==0.14.4
#10 68.94   Downloading dbt-postgres-0.14.4.tar.gz (7.6 kB)
#10 68.95   Preparing metadata (setup.py): started
#10 69.21   Preparing metadata (setup.py): finished with status 'done'
#10 69.38 Collecting psycopg2<2.8,>=2.7
#10 69.45   Downloading psycopg2-2.7.7.tar.gz (427 kB)
#10 69.46      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 427.5/427.5 kB 38.3 MB/s eta 0:00:00
#10 69.54   Preparing metadata (setup.py): started
#10 69.84   Preparing metadata (setup.py): finished with status 'error'
#10 69.85   error: subprocess-exited-with-error
#10 69.85   
#10 69.85   × python setup.py egg_info did not run successfully.
#10 69.85   │ exit code: 1
#10 69.85   ╰─> [23 lines of output]
#10 69.85       running egg_info
#10 69.85       creating /tmp/pip-pip-egg-info-9l56y9cr/psycopg2.egg-info
#10 69.85       writing /tmp/pip-pip-egg-info-9l56y9cr/psycopg2.egg-info/PKG-INFO
#10 69.85       writing dependency_links to /tmp/pip-pip-egg-info-9l56y9cr/psycopg2.egg-info/dependency_links.txt
#10 69.85       writing top-level names to /tmp/pip-pip-egg-info-9l56y9cr/psycopg2.egg-info/top_level.txt
#10 69.85       writing manifest file '/tmp/pip-pip-egg-info-9l56y9cr/psycopg2.egg-info/SOURCES.txt'
#10 69.85       
#10 69.85       Error: pg_config executable not found.
#10 69.85       
#10 69.85       pg_config is required to build psycopg2 from source.  Please add the directory
#10 69.85       containing pg_config to the $PATH or specify the full executable path with the
#10 69.85       option:
#10 69.85       
#10 69.85           python setup.py build_ext --pg-config /path/to/pg_config build ...
#10 69.85       
#10 69.85       or with the pg_config option in 'setup.cfg'.
#10 69.85       
#10 69.85       If you prefer to avoid building psycopg2 from source, please install the PyPI
#10 69.85       'psycopg2-binary' package instead.
#10 69.85       
#10 69.85       For further information please check the 'doc/src/install.rst' file (also at
#10 69.85       <<http://initd.org/psycopg/docs/install.html>>).
#10 69.85       
#10 69.85       [end of output]
#10 69.85   
#10 69.85   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 69.86 error: metadata-generation-failed
#10 69.86 
#10 69.86 × Encountered error while generating package metadata.
#10 69.86 ╰─> See above for output.
#10 69.86 
#10 69.86 note: This is an issue with the package mentioned above, not pip.
#10 69.86 hint: See above for details.
#10 70.05 
#10 70.05 [notice] A new release of pip is available: 23.0.1 -> 23.2.1
#10 70.05 [notice] To update, run: pip install --upgrade pip
#10 ERROR: process "/bin/sh -c if [ -f \"setup.py\" ]; then         pip install .;     fi" did not complete successfully: exit code: 1
------
 > [4/9] RUN if [ -f "setup.py" ]; then         pip install .;     fi:
69.86 error: metadata-generation-failed
69.86 
69.86 × Encountered error while generating package metadata.
69.86 ╰─> See above for output.
69.86 
69.86 note: This is an issue with the package mentioned above, not pip.
69.86 hint: See above for details.
70.05 
Notice: 70.05 [notice] A new release of pip is available: 23.0.1 -> 23.2.1
Notice: 70.05 [notice] To update, run: pip install --upgrade pip
------
Dockerfile:13
--------------------
  12 |     # copying all other files
  13 | >>> RUN if [ -f "setup.py" ]; then \
  14 | >>>         pip install .; \
  15 | >>>     fi
  16 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ -f \"setup.py\" ]; then         pip install .;     fi" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c if [ -f \"setup.py\" ]; then         pip install .;     fi" did not complete successfully: exit code: 1

Our

setup.py

file looks like this

rerunning the github action with debug logging enabled might also provide a better error

I’ll try with debug, but I think the error is sufficiently clear to see where the error is.

Given the time differences, do you have any other ideas? I really need to have this up and running by tomorrow.

If you can get an error other than an opaque "failed at pip install" you might be able to work around the failure with lifecycle hook scripts https://docs.dagster.io/dagster-cloud/deployment/serverless#using-a-different-base-image-or-using-native-dependencies or switch to using your own base image entirely

I think PEX will be the easiest route

@Todd de Quincey sorry for not looking at your original stacktrace in more detail it looks like

#10 69.85       If you prefer to avoid building psycopg2 from source, please install the PyPI
#10 69.85       'psycopg2-binary' package instead.

might be the real issue so maybe adding

psycopg2-binary

to your setup.py or using the lifecycle hook scripts to install postgres

Ahhhhh. I see why the Dagster serverless step was failing. Missing

dagster_cloud

dependency. I will add that today and see if that fixes it

Thanks for the breadcrumbs. This might have solved it 🤞

Hi @Joe, So fixing / using fast deploys did the trick. Thanks again for the assistance

Although, I need to go back to docker deploys to run a post script. But now I have this issue 😞 We’re slooooowly getting there though 🙂

Otherwise, unless I am misunderstanding, I’ll need to create my own image etc. Which I am trying to avoid at this stage since this is a PoC

the dockerfile doesn't modify any file permissions so if you haven't allowed execution of a file then i don't think it will run successfully

when you modify the file permission you should be able to commit and push it with that changed file metadata

python3.8 -m

might also be the issue (or whatever version you have configured)

actually I think dropping

python -m

should work

there will be no changes to commit if the file is already executable

which you can also verify with

ls -l ./dagster_cloud_pre_install.sh

Hi again @Joe, Sorry to bother you again. We are reeeeeeealy struggling to get the ssh port forwarding to work on Dagster Cloud correctly. We have no problems with it working locally. The post deployment script in the CD appears to run without a problem, yet, when we run the tasks in Dagster Cloud, we get the same old connection error, indicating that the ports aren’t forwarded (screenshot attached). I am presuming that post the deployment, the below script is actually run on the Dagster Cloud servers / in our docker containers? So the port forwarding (which the test in the below succeeded on CD) is applied to the actual instance/Serverless workers? I am thinking it might be easier to switch the Hybrid, or potentially self-host so that this is inside our VPC. Also a final note, I have ruled out that it is an IP whitelisting issue, as we temporarily allowed all IPV4 ip addresses. Any breadcrumbs would be most welcome

REDACTED

setting up ssh tunnels is something that you can do in your dagster resource code https://dagster.slack.com/archives/C01U954MEER/p1688624749164789?thread_ts=1688577139.518379&cid=C01U954MEER is an example of a user doing the same thing as you i think and dagster-ssh can help for reference

@Joe other said thread https://dagster.slack.com/archives/C01U954MEER/p1691448789451969?thread_ts=1691150052.375039&channel=C01U954MEER&message_ts=1691448789.451969

So long story short, I need to create an SSH resource to forward the ports when the dbt commands are invoked

If so, I’ll head back down that path

For the life of me, I cannot get this to work. This complains that

AttributeError: 'ResourceDefinition' object has no attribute 'get_tunnel'

, but then if I drop the

get_tunnel

, how/where do I define the tunnel details?

assets = with_resources(
    load_assets_from_dbt_project(
        profiles_dir=DBT_PROJECT_PATH,
        project_dir=DBT_PROFILES,
        else None,
        use_build_command=False,  # Delete when ready to launch to run tests
        display_raw_sql=True,
    ),
    {
        "dbt": dbt_cli_resource.configured(
            {
                "project_dir": DBT_PROJECT_PATH,
                "profiles_dir": DBT_PROFILES,
            },
        ),
        "ssh": ssh_resource.configured(
            {
                "remote_host": os.getenv("SSH_HOST"),
                "remote_port": 22,
                "username": os.getenv("SSH_USER"),
                "key_file": "~/.ssh/dagster_rsa",
            }
        ).get_tunnel(remote_port=5439, remote_host="localhost", local_port=5439),
    },
)