Alex Orlovskyi
08/29/2023, 12:25 PMdbt.get_dbt_client().run_operation(
macro="macro_name",
args={
"some_arg": some_value,
},
profile="profile_name",
full_refresh=False,
)
,
where dbt is DbtCliClientResource.
But with new dagster_dbt/core/resources_v2 and dbt = DbtCliResource I'm struggling to achieve same result as it was before (inside code snippet).
New DbtCliResource requires this code to be run inside @dbt_assets
, which is not preferable in my case, because I don't want to select any model from dbt project, just run the macro. Also it'll be better to keep separate py asset with it's own metadata and availability at dagster UI.
Any help towards this - appreciated.Alex Orlovskyi
08/29/2023, 12:33 PMrex
08/29/2023, 1:36 PMDbtCliResource
is a resource. It's not required to run inside @dbt_assets
. You can create an asset and reference it.
@asset
def my_dbt_macro(dbt: DbtCliResource):
dbt_macro_args = {"some_arg": "some_value"}
dbt.cli(["run-operation", "--args", json.dumps(dbt_macro_args)], manifest={}).wait()
Alex Orlovskyi
08/29/2023, 1:52 PMmanifest = Path(settings.DBT_PROJ_PATH, "target", "manifest.json")
dbt.cli(dbt_args, context=context, manifest=manifest).wait()
I'll get:
dagster._core.errors.DagsterInvariantViolationError: Expected to find dbt manifest metadata on asset asset/key, but did not. Did you pass in assets that weren't generated by load_assets_from_dbt_project, load_assets_from_dbt_manifest, or @dbt_assets?
same error for manifest={}
rex
08/29/2023, 1:56 PMAlex Orlovskyi
08/29/2023, 1:58 PMrex
08/29/2023, 1:59 PMmanifest={}
, we'll fix that with https://github.com/dagster-io/dagster/issues/15233rex
08/29/2023, 5:57 PM