Hi all, trying to get a POC up and running on dags...
# integration-dbt
m
Hi all, trying to get a POC up and running on dagster 1.3.10 using dbt 1.4.1. I have the existing dbt project that is working and when I reference the paths it appears to attempt to load but I get the error
2023-06-21 07:17:52 -0500 - dagster.code_server - INFO - Started Dagster code server for module SecurityMaster in process 2582919
/usr/local/lib/python3.9/site-packages/dagster/_core/workspace/context.py:610: UserWarning: Error loading repository location SecurityMaster:dagster._core.errors.DagsterInvalidDefinitionError: resource with key 'dbt' required by op 'run_dbt_281af' was not provided. Please provide a <class 'dagster._core.definitions.resource_definition.ResourceDefinition'> to key 'dbt', or change the required key to one of the following keys which points to an <class 'dagster._core.definitions.resource_definition.ResourceDefinition'>: ['io_manager']
my code is the simple example of f`rom dagster import Definitions, load_assets_from_modules`
from dagster_dbt import load_assets_from_dbt_project
from dagster import file_relative_path
from . import assets
DBT_PROJECT_PATH = file_relative_path(__file__, "../../../dbt-project/MyDbtProject")
DBT_PROFILES = file_relative_path(__file__, "../../../.dbt")
print(DBT_PROJECT_PATH)
dbt_assets = load_assets_from_dbt_project(
project_dir=DBT_PROJECT_PATH, profiles_dir=DBT_PROFILES, key_prefix=["CompanyName"],use_build_command=True
)
t
Hi Mike! It looks like you'll need to define a
dbt
resource so Dagster can manage dbt. Here's a link in our dbt tutorial that shows you how to do that:
Copy code
resources = {
    "dbt": DbtCliClientResource(
        project_dir=DBT_PROJECT_PATH,
        profiles_dir=DBT_PROFILES,
    ),
}

defs = Definitions(assets=load_assets_from_modules([assets]), resources=resources)
m
so it is complaining about ImportError: cannot import name 'DBT_PROFILES' from partially initialized module 'ProjectMaster' (most likely due to a circular import) (/home/mike.needham/dagster/ProjectMaster/ProjectMaster/__init__.py)
I have an existing dbt project that is being used and do not want to mess with it so just trying to point to it
t
Thankfully, it looks like your new error message isn't about dbt or Dagster, but rather a Python module dependency issue. are you trying to import
DBT_PROFILES
from somewhere into your root-level
__init__.py
?
m
that was it, tracked it down to one of the imports
thanks for the help
t
blob salute 🎉