https://dagster.io/ logo
Title
d

David Jayatillake

11/20/2022, 12:40 AM
Hi 👋 https://github.com/metaplane/metaplane-dagster/pull/8 anyone know why I’m getting this error where it’s saying there is no dbt_project.yml file in the directory?
I got this to work on my dev codespace by deleting the submodule and bringing it back in by running
git submodule add <https://github.com/metaplane/metaplane-dbt> metaplane-dbt
how do I get something like this to run in my dbt cloud build?
I also had to install the snowflake dbt-adapter which I added to the requirements.txt, but it fails unless dbt deps is run too.
s

sandy

11/21/2022, 4:11 PM
@rex mind taking a look?
r

rex

11/21/2022, 6:30 PM
Hey David! With 1.1.0, you can essentially do something like this to load your assets from a dbt cloud job:
from dagster import repository
from dagster_dbt import dbt_cloud_resource, load_assets_from_dbt_cloud_job

DBT_CLOUD_JOB_ID = <Your job id>

dbt_cloud = dbt_cloud_resource.configured(
    {
        "auth_token": {"env": "DBT_CLOUD_API_TOKEN"},
        "account_id": {"env": "DBT_CLOUD_ACCOUNT_ID"},
    }
)

dbt_cloud_assets = load_assets_from_dbt_cloud_job(
    dbt_cloud=dbt_cloud, job_id=DBT_CLOUD_JOB_ID
)


@repository
def dbt_cloud_sandbox():
    return [dbt_cloud_assets]
we assume that your dbt cloud job has a
dbt run
or
dbt build
and we generate the project dependencies from that. The dbt cloud job runs
dbt deps
automatically, so you don’t have to worry about that
d

David Jayatillake

11/21/2022, 6:32 PM
Oh this is for dbt core
r

rex

11/21/2022, 6:41 PM
oh, what did you mean by
dbt cloud build
here?
how do I get something like this to run in my dbt cloud build
d

David Jayatillake

11/21/2022, 6:56 PM
oh sorry 🤦 I meant dagster cloud build
so that when dagster cloud builds, it runs dbt deps first thing on my dbt project so that it has everything it needs
it’s the sort of thing that should just happen rather than configure, as there is no cost if you don’t use dbt packages but if you do then it will definitely break.
r

rex

11/21/2022, 10:32 PM
ah ok. in that case,
dbt deps
should be run in the Dockerfile that contains your Dagster code location. This way, dependencies are installed only once. Dagster shouldn’t need to install dependencies every single time a run is created. All dependencies should present once the run starts.
👍 1