Hello, I have recently switched to using Dagster ...
# ask-community
Hello, I have recently switched to using Dagster Cloud from Standard. Previously I was using the dbt_cli_resource to run dbt. I had configured it to find the profiles.yml and the dbt source on the local file system like so:
Copy code
"dbt": dbt_cli_resource.configured(
            "project_dir": '../relative/path/dbt',
            "profiles_dir": '../relative/path'
Now that I'm running on cloud I'm seeing the following error:
Copy code
Encountered an error:
Runtime Error
fatal: Invalid --project-dir flag. Not a dbt project. Missing dbt_project.yml file
It seems like the dbt folder is not being packaged with my dagster source and deployed to cloud where it can run. Is there a way to deploy the dbt with the dagster code, or do I need to run dbt with dbt-cloud? Thanks, in advance
hi @Don! this is definitely supported, but you'll want to use absolute paths rather than relative paths, as we don't make any guarantees on which working directory a command will be invoked from. The general pattern we recommend is using file_relative_path, which will turn your relative path into an absolute one. see: https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster/part-three#step-1-create-upstream-dagster-assets
Thanks, I'll give that a try
It looks like my example was wrong and I was already using file_relative_path
Copy code
DBT_PROJECT_DIR = file_relative_path(__file__, "../../dbt")
DBT_PROFILES_DIR = file_relative_path(__file__, "../..")
however, my dbt project lives at the same level as my dagster source. so it's like this project -> dbt -> my_dagster -> config job_config.py (this is where the file_relative_path is being used) repository.py dagster_cloud.yaml
my dagster_cloud.yaml looks like this
Copy code
  - location_name: my_dagster
      package_name: my_dagster
I am guessing that the dbt code is not getting packaged with my dagster code
is there a way to include the whole project in my cloud package or should I move the dbt code into my_dagster?
@owen Thanks for your help, I updated with more clarification
Hey Don - does this discussion answer the question about how to include the dbt project? https://github.com/dagster-io/dagster/discussions/13767
@daniel, thanks. I will give this a try. So basically, the dbt should be inside the dagster folder, but it can either be done in the repo or during CI/CD