After upgrading dagster-dbt to

0.19.2

I started getting an error saying dbt can’t find a

dbt_project.yml

inside the

--project-dir

, is this a known issue that will be fixed in the next release? I didn’t change anything else besides the upgrade. Tested on dbt-core

1.4.2

and

1.4.6

, running

ls $PROJECT_DIR

inside the docker image shows the dbt_project file correctly and the dbt command seems to be parsed correctly by dagster, so I’m at a loss here. Parsed command:

dbt --no-use-colors --log-format json ls --project-dir dp_dbthub --profiles-dir dp_dbthub/profiles --select * --output json

hi @Vinnie! thanks for the report -- I think I know what's causing this.

dbt-core

has historically been fairly inconsistent on how it handles relative paths passed into the command, and most recently this regression in

dbt-core

1.5.0 prompted us to make a change such that commands would be executed in the project directory. in cases where the project-dir is a relative path, rather than an absolute path, this would cause the issue you're seeing. so the immediate fix would be to configure your dbt resource with an absolute path, but this is definitely a bug that we should fix. just created a ticket here, which should be solvable for next week's release

Thanks for the explanation! Will give it a go later this week

Hi @Vinnie This is a bit of a rookie question. I'm experiencing an issue since upgrading to dbt-core==1.5.0 and i'm wondering if my issue might be similar to the one you experienced when using relative paths. Anyway i'm trying to change my project to reference absolute paths instead of relative paths for the DBT_PROJECT_PATH & DBT_PROFILES. I'm not having much luck making this change and was wondering how you did it? Below is a code block of what I tried which does not work and gives error: Runtime Error fatal: Invalid --project-dir flag. Not a dbt project. Missing dbt_project.yml file The commented out bit is the relative approach which works fine...

from pathlib import Path

#DBT_PROJECT_PATH = file_relative_path(__file__, "../../dbt_project")
DBT_PROJECT_PATH = str(Path(__file__).absolute().parent.parent / 'dbt_project')

hi @Ian Venter! the commented-out bit actually looks correct to me (

file_relative_path

is actually creating an absolute path out of the file-relative path) are you hitting the same error in both of those forms? if so, can you confirm the dagster-dbt version you're using, and what the last versions of dbt-core/dagster-dbt that worked for you were?

Hi @owen , I’ve tagged u on my original issue.

gotcha, thanks for the extra context! I was also unable to replicate this error, so I would stick with rex's original recommendation. If you're able to get some steps to reproduce, I'm happy to look a bit deeper into this. The closest I can get to an explanation is that your dbt_project.yml is somehow in a format that dbt-core doesn't like (it's almost like all of it under a

config:

block, which doesn't make a ton of sense)