Tried searching but haven't found any results here in slack or online. I am using pycopg2 for some things and dagster is struggling to build

#10 3.541   × python setup.py egg_info did not run successfully.
#10 3.541   │ exit code: 1
#10 3.541   ╰─> [23 lines of output]
#10 3.541       running egg_info
#10 3.541       creating /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info
#10 3.541       writing /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/PKG-INFO
#10 3.541       writing dependency_links to /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/dependency_links.txt
#10 3.541       writing top-level names to /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/top_level.txt
#10 3.541       writing manifest file '/tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/SOURCES.txt'
#10 3.541       
#10 3.541       Error: pg_config executable not found.
#10 3.541       
#10 3.541       pg_config is required to build psycopg2 from source.  Please add the directory
#10 3.541       containing pg_config to the $PATH or specify the full executable path with the
#10 3.541       option:
#10 3.541       
#10 3.541           python setup.py build_ext --pg-config /path/to/pg_config build ...
#10 3.541       
#10 3.541       or with the pg_config option in 'setup.cfg'.
#10 3.541       
#10 3.541       If you prefer to avoid building psycopg2 from source, please install the PyPI
#10 3.541       'psycopg2-binary' package instead.
#10 3.541       
#10 3.541       For further information please check the 'doc/src/install.rst' file (also at
#10 3.541       <<https://www.psycopg.org/docs/install.html>>).
#10 3.541       
#10 3.541       [end of output]
#10 3.541   
#10 3.541   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 3.544 error: metadata-generation-failed
#10 3.544 
#10 3.544 × Encountered error while generating package metadata.
#10 3.544 ╰─> See above for output.
#10 3.544 
#10 3.544 note: This is an issue with the package mentioned above, not pip.
#10 3.544 hint: See above for details.

any ideas would be greatly appreciated!

have you tried using

psycopg2-binary

instead of

psycopg2

?

I havent yet. From what I read, psycopg2 is preferable when interacting with database servers. The former is supposed to be very basic psql and python interactions. I've been ysing psycopg2 locally and haven't had any issues.

ah I see. have you tried installing

libpq-dev

? (https://stackoverflow.com/a/12037133/6297800)

This is already installed on my local machine

and psycopg2 is working in other projects on your local machine, but it just won't build when you try to install dagster locally?

yep, exactly

how are you installing Dagster?

im using the serverless deploy, importing my own project

I guess I'm still a bit confused - are you having issues just getting Dagster to install at all locally, or is it when you're deploying that you're seeing issues in CI/CD, or both? I'm not super familiar with dagster serverless but in that context it might make sense to try adding

dagster-postgres

sorry, its when Im deploying where I see issues in the CI/CD -

ah okay, yeah I suspect in that case that

libpq-dev

isn't installed in the container that's being built. it seems likely that you'll need to provide your own docker image which includes

libpq-dev

, or disabling PEX-based deploys and providing a

dagster_cloud_pre_install.sh

script to install

libpq-dev

yeah, its just reading and writing...do you think its better to use binary insterad?

I think it'd be easier, but it doesn't seem like it'd be super hard to modify your serverless deployment workflow to user a custom docker image or pre-install script - I do think your deploys would be a decent amount slower as a result though. really up to you

the latter option of just moving to binary fixed thus