https://dagster.io/ logo
#ask-community
Title
# ask-community
j

jan

04/26/2022, 3:09 PM
In general, is there any imperative instruction on how to use Libraries? The docs list them, but I wasn’t able to find a working implementation tutorial.
p

prha

04/26/2022, 4:43 PM
Hi Jan! Dagster has quite a number of pluggable components, so each library is potentially invoked in a different way, depending on what functionality you’re trying to achieve. Are there some libraries that you’re particularly interested in where the specific documentation could be improved?
j

jan

04/27/2022, 7:31 AM
Hi @prha, I’m not quite thinking of what you’re describing, I think. And I gotta say, possibly this is just me not having studied the docs deeply enough. So I’m not looking for functional documentation of these components, they often already exist as these are standalone projects, right? What I ran into (and from studying the Slack archives some people also experienced this, for others it seemed to work automagically) is how to properly operationalize the modules in Dagster so they can be used in a file. Basically I ran into 1. pip install component, i.e. dagster-airbyte: Check 2. ????? 3. use the component as
from dagster_airbyte import foo
4.
Module not found
error when trying
dagit -f airbyte_foo.py
So then it turns out files must not be in the same dir as the virtual env holding the python installed component for modules to be parsed properly. This, to me, was just not transparent.
@prha to give you a more specific example: I just cloned the dagster repo and went into the
examples/hacker_news
directory. It has a high level
setup.py
as well as a
workspace.yaml
. So again, my steps are 1. create venv, run
pip install -e .
, setup.py gets executed. 2. run
dagit
3. Error:
Module not found: dagster_dbt
So again, dagster seems not to be able to parse modules correctly despite a present setup.py containing dagster-dbt as well as a workspace.yaml and no further context as to what’s the rootcause of this mistake is provided.