Hi folks, I'm trying out the new DBT integration f...
# ask-community
s
Hi folks, I'm trying out the new DBT integration following from yesterdays demo, I'm at this stage - https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster/load-dbt-models#step-5-understand-the-python-code-in-your-dagster-project Previously I had specified the location of my profiles directory like so:
Copy code
dbt_assets = load_assets_from_dbt_project(
    project_dir=DBT_PROJECT_DIR,
    profiles_dir=DBT_PROFILES_DIR,
    key_prefix=['aa']
)
However I don't see that in
DbtCliResource
, I had tried setting
--profiles-dir
in
global_config_flags
to be told that option is not available. Running a raw DBT command, I can see the option is not there either:
Copy code
(venv) ➜  dbt_project git:(airbyte_integration) ✗ dbt --profiles-dir a                                                                              
Usage: dbt [OPTIONS] COMMAND [ARGS]...
Try 'dbt -h' for help.

Error: No such option: --profiles-dir
(venv) ➜  dbt_project git:(airbyte_integration) ✗
It is still shown as an option on the DBT website (using the
DBT_PROFILES_DIR
env var still works) - https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles#advanced-customizing-a-profile-directory I'm wondering if this is a regression on the DBT side or if I'm missing something, thanks. Edit: not a DBT regression, it was me forgetting that you need to use the switch in conjunction with another command, eg
dbt build --profiles-dir your_dir
🤖 1
r
• This has been previously discussed here: https://dagster.slack.com/archives/C04CW71AGBW/p1690578876840239?thread_ts=1690384210.355069&cid=C04CW71AGBW • Looks like it isn’t intuitive for folks to have their
profiles.yml
in the root of their directory, so we’ve added it back as configuration to
DbtCliResource
! https://github.com/dagster-io/dagster/pull/15722
s
Looks like it isn’t intuitive for folks to have their
profiles.yml
Agreed. In my case I'm putting references to JSON files which BigQuery uses for authentication purposes. Not sensitive in itself, though no need to be in the repo. I take it that change you made will be on the next release? I see its merged
Also edited my original post regarding my mistaken observation on DBT regression
r
yup! will be out this thursday
s
Great cheers