If I use a

DbtCliResource

to compile a dbt project the artefacts including the manifest end up in a subfolder, e.g.,

8ec9053/manifest.json

. Is the subfoldering a Dagster thing, or a new dbt thing?

It's a Dagster thing. We put the artifacts in subfolders so that if you're running multiple commands, the artifacts don't clobber each other

ah, good to know.

If you don't care about that, you can just hard code the target directory in which the artifacts are created when calling `DbtCliResource.cli(...)`: https://docs.dagster.io/_apidocs/libraries/dagster-dbt#dagster_dbt.DbtCliResource.cli

• target_path (Optional[Path]) – An explicit path to a target folder to use to store and retrieve dbt artifacts when running a dbt CLI command. If not provided, a unique target path will be generated.

cool, that answers my next question. thanks rex!

yup!

It's similar to the dagster_dbt example. In a local dev environment using local k8s (minikube) I want dagster to trigger an accurate manifest to be generated when the code deployment is refreshed, hence the parse.

Ah got it. FWIW, we default to creating the manifest in a static directory now, when running

dagster-dbt project scaffold

. This was after some feedback from the community: https://github.com/dagster-io/dagster/pull/18501/files The relevant tidbit:

if os.getenv("DAGSTER_DBT_PARSE_PROJECT_ON_LOAD"):
    dbt_manifest_path = (
        dbt.cli(
            ["--quiet", "parse"],
            target_path=Path("target"),
        )
        .wait()
        .target_path.joinpath("manifest.json")
    )

The real issue I've got (and how I spotted the foldering) is that I'm trying to integrate this into an existing dbt project with a typer CLI that allows the user to

dbt clone

into a dev database from a production manifest that's backed up on GCS. The CLI currently sits in the same project as the dagster assets and the

@dbt_asset()

decorator was forcing a compile on every command because the target path was changing.