Tried searching but haven't found any results here...
# ask-community
k
Tried searching but haven't found any results here in slack or online. I am using pycopg2 for some things and dagster is struggling to build
Copy code
#10 3.541   × python setup.py egg_info did not run successfully.
#10 3.541   │ exit code: 1
#10 3.541   ╰─> [23 lines of output]
#10 3.541       running egg_info
#10 3.541       creating /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info
#10 3.541       writing /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/PKG-INFO
#10 3.541       writing dependency_links to /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/dependency_links.txt
#10 3.541       writing top-level names to /tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/top_level.txt
#10 3.541       writing manifest file '/tmp/pip-pip-egg-info-iekbphju/psycopg2.egg-info/SOURCES.txt'
#10 3.541       
#10 3.541       Error: pg_config executable not found.
#10 3.541       
#10 3.541       pg_config is required to build psycopg2 from source.  Please add the directory
#10 3.541       containing pg_config to the $PATH or specify the full executable path with the
#10 3.541       option:
#10 3.541       
#10 3.541           python setup.py build_ext --pg-config /path/to/pg_config build ...
#10 3.541       
#10 3.541       or with the pg_config option in 'setup.cfg'.
#10 3.541       
#10 3.541       If you prefer to avoid building psycopg2 from source, please install the PyPI
#10 3.541       'psycopg2-binary' package instead.
#10 3.541       
#10 3.541       For further information please check the 'doc/src/install.rst' file (also at
#10 3.541       <<https://www.psycopg.org/docs/install.html>>).
#10 3.541       
#10 3.541       [end of output]
#10 3.541   
#10 3.541   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 3.544 error: metadata-generation-failed
#10 3.544 
#10 3.544 × Encountered error while generating package metadata.
#10 3.544 ╰─> See above for output.
#10 3.544 
#10 3.544 note: This is an issue with the package mentioned above, not pip.
#10 3.544 hint: See above for details.
any ideas would be greatly appreciated!
🤖 1
dagster bot responded by community 1
z
have you tried using
psycopg2-binary
instead of
psycopg2
?
k
I havent yet. From what I read, psycopg2 is preferable when interacting with database servers. The former is supposed to be very basic psql and python interactions. I've been ysing psycopg2 locally and haven't had any issues.
z
ah I see. have you tried installing
libpq-dev
? (https://stackoverflow.com/a/12037133/6297800)
pretty sure that worked for me a while ago when I ran into this error
k
This is already installed on my local machine
z
and psycopg2 is working in other projects on your local machine, but it just won't build when you try to install dagster locally?
k
yep, exactly
z
how are you installing Dagster?
k
im using the serverless deploy, importing my own project
Copy code
dagit                  1.1.14
dagster                1.1.14
dagster-graphql        1.1.14
z
I guess I'm still a bit confused - are you having issues just getting Dagster to install at all locally, or is it when you're deploying that you're seeing issues in CI/CD, or both? I'm not super familiar with dagster serverless but in that context it might make sense to try adding
dagster-postgres
k
sorry, its when Im deploying where I see issues in the CI/CD -
z
ah okay, yeah I suspect in that case that
libpq-dev
isn't installed in the container that's being built. it seems likely that you'll need to provide your own docker image which includes
libpq-dev
, or disabling PEX-based deploys and providing a
dagster_cloud_pre_install.sh
script to install
libpq-dev
but adding
psycopg2-binary
should "just work". if you're just using psycopg2 for some basic postgres interactions I don't think you have much to worry about using it, it's very stable
k
yeah, its just reading and writing...do you think its better to use binary insterad?
instead*
z
I think it'd be easier, but it doesn't seem like it'd be super hard to modify your serverless deployment workflow to user a custom docker image or pre-install script - I do think your deploys would be a decent amount slower as a result though. really up to you
k
the latter option of just moving to binary fixed thus
this*
thanks @Zach appreciate the assist