Hi, I’m trying to work through the dbt tutorial he...
# ask-community
d
Hi, I’m trying to work through the dbt tutorial here: https://github.com/dagster-io/dagster/tree/master/examples/tutorial_dbt_dagster#getting-started When I run
dagit
, I get
ModuleNotFoundError: No module named 'dagster_dbt'
Copy code
$ pip freeze | grep dagster-dbt
dagster-dbt==0.17.6
Any idea what’s going on here?
r
There’s a tutorial linked in that readme: https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster#prerequisites
• Dependencies for the following libraries:
◦ dagster-dbt. This library allows you to integrate dbt with Dagster.
◦ dbt-duckdb. This tutorial uses DuckDB as the database backing dbt.
◦ dagster-duckdb. This library provides a resource that enables the materialization of Dagster assets as DuckDB tables.
◦ dagster-duckdb-pandas. This library allows you to store pandas DataFrames in DuckDB.
◦ pandas. This tutorial uses pandas to fetch raw data.
◦ plotly. This tutorial uses plotly to create a histogram asset.
you’ll need to pip install these dependencies.
d
Hey rex, I’ve installed all of those
I ran
pip install -e ".[dev]"
which installed all of those, here’s the output of `pip freeze`:
Copy code
$ pip freeze                                                     9:25
agate==1.6.3
alembic==1.8.1
aniso8601==9.0.1
anyio==3.6.2
attrs==22.1.0
Babel==2.11.0
backoff==2.2.1
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
colorama==0.4.5
coloredlogs==14.0
croniter==1.3.8
dagit==1.1.6
dagster==1.1.6
dagster-dbt==0.17.6
dagster-duckdb==0.17.6
dagster-duckdb-pandas==0.17.6
dagster-graphql==1.1.6
dbt-core==1.3.1
dbt-duckdb==1.3.3
dbt-extractor==0.4.1
docstring-parser==0.15
duckdb==0.6.1
exceptiongroup==1.0.4
fsspec==2022.11.0
future==0.18.2
gql==3.4.0
graphene==3.2.1
graphql-core==3.2.3
graphql-relay==3.2.0
greenlet==2.0.1
grpcio==1.47.2
grpcio-health-checking==1.43.0
h11==0.14.0
hologram==0.0.15
httptools==0.5.0
humanfriendly==10.0
idna==3.4
iniconfig==1.1.1
isodate==0.6.1
Jinja2==3.1.2
jsonschema==3.2.0
leather==0.3.4
Logbook==1.5.3
Mako==1.2.4
MarkupSafe==2.1.1
mashumaro==3.0.4
minimal-snowplow-tracker==0.0.2
msgpack==1.0.4
multidict==6.0.3
networkx==2.8.8
numpy==1.23.5
packaging==21.3
pandas==1.5.2
parsedatetime==2.4
pathspec==0.9.0
pendulum==2.1.2
plotly==5.11.0
pluggy==1.0.0
protobuf==3.20.3
pycparser==2.21
pyparsing==3.0.9
pyrsistent==0.19.2
pytest==7.2.0
python-dateutil==2.8.2
python-dotenv==0.21.0
python-slugify==7.0.0
pytimeparse==1.1.8
pytz==2022.6
pytzdata==2020.1
PyYAML==6.0
requests==2.28.1
requests-toolbelt==0.10.1
six==1.16.0
sniffio==1.3.0
SQLAlchemy==1.4.45
sqlparse==0.4.3
starlette==0.23.1
tabulate==0.9.0
tenacity==8.1.0
text-unidecode==1.3
tomli==2.0.1
toposort==1.7
tqdm==4.64.1
# Editable install with no version control (tutorial-dbt-dagster==0.0.0)
-e /Users/dean/dev/my-dagster-project
typing_extensions==4.4.0
universal_pathlib==0.0.21
urllib3==1.26.13
uvicorn==0.20.0
uvloop==0.17.0
watchdog==2.2.0
watchfiles==0.18.1
websockets==10.4
Werkzeug==2.2.2
yarl==1.8.2
r
I’m unable to reproduce this. Are you using a virtualenv?
Copy code
internal-3.8.7 ❯ dagster project from-example --name my-dagster-project --example tutorial_dbt_dagster

Downloading example 'tutorial_dbt_dagster'. This may take a while.
Success! Created my-dagster-project at /Users/rexledesma/elementl/my-dagster-project.

~/elementl
internal-3.8.7 ❯ cd my-dagster-project

~/elementl/my-dagster-project
internal-3.8.7 ❯ pyenv virtualenv 3.9.13 dbt-project-3.9.13
Looking in links: /var/folders/yb/qqcnzws17_v917wn0t26nsfw0000gn/T/tmpi7__72gu
Requirement already satisfied: setuptools in /Users/rexledesma/.pyenv/versions/3.9.13/envs/dbt-project-3.9.13/lib/python3.9/site-packages (58.1.0)
Requirement already satisfied: pip in /Users/rexledesma/.pyenv/versions/3.9.13/envs/dbt-project-3.9.13/lib/python3.9/site-packages (22.0.4)

~/elementl/my-dagster-project 11s
internal-3.8.7 ❯ pyenv activate dbt-project-3.9.13

~/elementl/my-dagster-project
dbt-project-3.9.13 ❯ pip install -e ".[dev]"
...

~/elementl/my-dagster-project
dbt-project-3.9.13 ❯ cd tutorial_finished

~/elementl/my-dagster-project/tutorial_finished
dbt-project-3.9.13 ❯ dagit
d
I am using a virtualenv…
Does the version of dagster I use for
dagster project from-example
matter?
Actually ignore that, I’m using 1.1.6 there as well
Would you mind running
pip freeze
and sharing the output?
r
Copy code
~/elementl/my-dagster-project/tutorial_finished
dbt-project-3.9.13 ❯ pip freeze
agate==1.6.3
alembic==1.8.1
aniso8601==9.0.1
anyio==3.6.2
attrs==22.1.0
Babel==2.11.0
backoff==2.2.1
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
colorama==0.4.5
coloredlogs==14.0
croniter==1.3.8
dagit==1.1.6
dagster==1.1.6
dagster-dbt==0.17.6
dagster-duckdb==0.17.6
dagster-duckdb-pandas==0.17.6
dagster-graphql==1.1.6
dbt-core==1.3.1
dbt-duckdb==1.3.3
dbt-extractor==0.4.1
docstring-parser==0.15
duckdb==0.6.1
exceptiongroup==1.0.4
fsspec==2022.11.0
future==0.18.2
gql==3.4.0
graphene==3.2.1
graphql-core==3.2.3
graphql-relay==3.2.0
greenlet==2.0.1
grpcio==1.47.2
grpcio-health-checking==1.43.0
h11==0.14.0
hologram==0.0.15
httptools==0.5.0
humanfriendly==10.0
idna==3.4
iniconfig==1.1.1
isodate==0.6.1
Jinja2==3.1.2
jsonschema==3.2.0
leather==0.3.4
Logbook==1.5.3
Mako==1.2.4
MarkupSafe==2.1.1
mashumaro==3.0.4
minimal-snowplow-tracker==0.0.2
msgpack==1.0.4
multidict==6.0.3
networkx==2.8.8
numpy==1.23.5
packaging==21.3
pandas==1.5.2
parsedatetime==2.4
pathspec==0.9.0
pendulum==2.1.2
plotly==5.11.0
pluggy==1.0.0
protobuf==3.20.3
pycparser==2.21
pyparsing==3.0.9
pyrsistent==0.19.2
pytest==7.2.0
python-dateutil==2.8.2
python-dotenv==0.21.0
python-slugify==7.0.0
pytimeparse==1.1.8
pytz==2022.6
pytzdata==2020.1
PyYAML==6.0
requests==2.28.1
requests-toolbelt==0.10.1
six==1.16.0
sniffio==1.3.0
SQLAlchemy==1.4.45
sqlparse==0.4.3
starlette==0.23.1
tabulate==0.9.0
tenacity==8.1.0
text-unidecode==1.3
tomli==2.0.1
toposort==1.7
tqdm==4.64.1
# Editable install with no version control (tutorial-dbt-dagster==0.0.0)
-e /Users/rexledesma/elementl/my-dagster-project
typing_extensions==4.4.0
universal_pathlib==0.0.21
urllib3==1.26.13
uvicorn==0.20.0
uvloop==0.17.0
watchdog==2.2.0
watchfiles==0.18.1
websockets==10.4
Werkzeug==2.2.2
yarl==1.8.2
can you validate that your virtualenv are in your
$PATH
?
You should see something like
:/Users/rexledesma/.pyenv/shims:/Users/rexledesma/.pyenv/bin:
when you do
Copy code
echo $PATH
This is so that the correct interpreter can be used - just wanted to confirm this
d
Sure thing, I created a virtual env like so:
Copy code
python3 -m venv venv
. venv/bin/activate
echo $PATH
/Users/dean/dev/my-dagster-project/venv/bin:...
Let me try creating a pyenv virtualenv like you’ve done, just to eliminate any variables
Hey thanks for trying to reproduce this, it’s working now! When I installed
pyenv-virtualenv
it also upgraded
pyenv
and I had to reinstall all of my python versions. I can’t reproduce it either anymore 🤷
r
glad it’s working for you!