https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
a

Alex Orlovskyi

08/29/2023, 12:25 PM
Hi. With old dagster_dbt/core/resources i could have following snippet inside python asset:
dbt.get_dbt_client().run_operation(
macro="macro_name",
args={
"some_arg": some_value,
},
profile="profile_name",
full_refresh=False,
)
, where dbt is DbtCliClientResource. But with new dagster_dbt/core/resources_v2 and dbt = DbtCliResource I'm struggling to achieve same result as it was before (inside code snippet). New DbtCliResource requires this code to be run inside
@dbt_assets
, which is not preferable in my case, because I don't want to select any model from dbt project, just run the macro. Also it'll be better to keep separate py asset with it's own metadata and availability at dagster UI. Any help towards this - appreciated.
@rex maybe you can advise something in this situation?
r

rex

08/29/2023, 1:36 PM
DbtCliResource
is a resource. It's not required to run inside
@dbt_assets
. You can create an asset and reference it.
Copy code
@asset
def my_dbt_macro(dbt: DbtCliResource):
    dbt_macro_args = {"some_arg": "some_value"}

    dbt.cli(["run-operation", "--args", json.dumps(dbt_macro_args)], manifest={}).wait()
🤔 1
a

Alex Orlovskyi

08/29/2023, 1:52 PM
@rex if i pass manifest like this
Copy code
manifest = Path(settings.DBT_PROJ_PATH, "target", "manifest.json")

dbt.cli(dbt_args, context=context, manifest=manifest).wait()
I'll get:
dagster._core.errors.DagsterInvariantViolationError: Expected to find dbt manifest metadata on asset asset/key, but did not. Did you pass in assets that weren't generated by load_assets_from_dbt_project, load_assets_from_dbt_manifest, or @dbt_assets?
same error for
manifest={}
r

rex

08/29/2023, 1:56 PM
Could you remove the context?
a

Alex Orlovskyi

08/29/2023, 1:58 PM
@rex now it works) thanks a lot for help! 🎉 ty spinny
r

rex

08/29/2023, 1:59 PM
Awesome. We can add an example for this in the API docs. Re: the ergonomics with
manifest={}
, we'll fix that with https://github.com/dagster-io/dagster/issues/15233
hughie ohno 1
https://github.com/dagster-io/dagster/pull/16179 here's an example for the future
ty spinny 1
3 Views