https://dagster.io/ logo
#ask-community
Title
# ask-community
b

Ben Gatewood

03/29/2022, 4:16 AM
Anybody able to shed any light on this one:
Copy code
Traceback (most recent call last): File "/usr/local/bin/dagster", 
line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/dagster/cli/__init__.py", 
line 50, in main cli(auto_envvar_prefix=ENV_PREFIX) # pylint:disable=E1123 File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 1128, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 1053, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.8/site-packages/click/core.py", 
line 754, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/dagster/cli/api.py", 
line 52, in execute_run_command DagsterInstance.from_ref(args.instance_ref) File "/usr/local/lib/python3.8/site-packages/dagster/core/instance/__init__.py", 
line 423, in from_ref run_storage=instance_ref.run_storage, File "/usr/local/lib/python3.8/site-packages/dagster/core/instance/ref.py", 
line 240, in run_storage return self.run_storage_data.rehydrate() File "/usr/local/lib/python3.8/site-packages/dagster/serdes/config_class.py", 
line 56, in rehydrate module = importlib.import_module(self.module_name) File "/usr/local/lib/python3.8/importlib/__init__.py", 
line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", 
line 1014, in _gcd_import File "<frozen importlib._bootstrap>", 
line 991, in _find_and_load File "<frozen importlib._bootstrap>", 
line 961, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", 
line 219, in _call_with_frames_removed File "<frozen importlib._bootstrap>", 
line 1014, in _gcd_import File "<frozen importlib._bootstrap>", 
line 991, in _find_and_load File "<frozen importlib._bootstrap>", 
line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", 
line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", 
line 843, in exec_module File "<frozen importlib._bootstrap>", 
line 219, in _call_with_frames_removed File "/usr/local/lib/python3.8/site-packages/dagster_postgres/__init__.py", 
line 3, in <module> from .event_log import PostgresEventLogStorage File "/usr/local/lib/python3.8/site-packages/dagster_postgres/event_log/__init__.py", 
line 1, in <module> from .event_log import PostgresEventLogStorage File "/usr/local/lib/python3.8/site-packages/dagster_postgres/event_log/event_log.py", 
line 27, in <module> from ..pynotify import await_pg_notifications File "/usr/local/lib/python3.8/site-packages/dagster_postgres/pynotify.py", 
line 40, in <module> from .utils import get_conn File "/usr/local/lib/python3.8/site-packages/dagster_postgres/utils.py", 
line 13, in <module> from dagster.core.definitions.policy import Backoff, Jitter, calculate_delay ImportError: cannot import name 'calculate_delay' from 'dagster.core.definitions.policy' (/usr/local/lib/python3.8/site-packages/dagster/core/definitions/policy.py)
d

daniel

03/29/2022, 2:49 PM
Hi Ben - this looks to me like your dagster-postgres package version is different than your dagster package version - there's a pin that's intended to enforce that their versions are exactly the same
If you run 'pip freeze | grep dagster' that should confirm whether that's the case
b

Ben Gatewood

03/29/2022, 10:57 PM
Thanks, Daniel - that's weird. Would this be in my user code repository image? Because all the other ones are the official dagster images
d

daniel

03/29/2022, 10:58 PM
Yeah, this looks like your image that's used when launching a run
b

Ben Gatewood

03/29/2022, 11:00 PM
so dagster-postgres in my container is
Copy code
dagster-postgres==0.14.6
Which matches the dagster version I specified when I built it so that makes sense
OK how on earth would this happen:
Copy code
dagster==0.14.2
dagster-aws==0.14.2
dagster-celery==0.14.6
dagster-celery-k8s==0.14.6
dagster-gcp==0.14.6
dagster-graphql==0.14.2
dagster-k8s==0.14.6
dagster-pandas==0.14.6
dagster-postgres==0.14.6
?
d

daniel

03/29/2022, 11:03 PM
Did you build this with a Dockerfile, could you share that?
b

Ben Gatewood

03/29/2022, 11:04 PM
Yeah I copied the example one from the docs and tweaked it a bit for my purposes. Looks like this:
Copy code
FROM python:3.8-slim

ARG DAGSTER_VERSION

# ==> Add Dagster layer
RUN \
    pip install \
        dagster==${DAGSTER_VERSION} \
        dagster-postgres==${DAGSTER_VERSION} \
        dagster-celery[flower,redis,kubernetes]==${DAGSTER_VERSION} \
        dagster-aws==${DAGSTER_VERSION} \
        dagster-gcp==${DAGSTER_VERSION} \
        dagster-k8s==${DAGSTER_VERSION} \
        dagster-celery-k8s==${DAGSTER_VERSION} \
# Cleanup
    &&  rm -rf /var \
    &&  rm -rf /root/.cache  \
    &&  rm -rf /usr/lib/python2.7 \
    &&  rm -rf /usr/lib/x86_64-linux-gnu/guile

RUN mkdir -p /opt/dagster/app

WORKDIR /opt/dagster/app

COPY mlb /opt/dagster/app/
COPY requirements.txt /opt/dagster/app

RUN pip install -r requirements.txt
Then built it with this build arg:
Copy code
--build-arg DAGSTER_VERSION=0.14.6
d

daniel

03/29/2022, 11:07 PM
What about the contents of that requirements.txt? Any dagster stuff in there?
b

Ben Gatewood

03/29/2022, 11:08 PM
Argh yup there it is facepalm
Sorry!
d

daniel

03/29/2022, 11:09 PM
No worries!
b

Ben Gatewood

03/29/2022, 11:10 PM
Thanks for the help!
condagster 1
2 Views