https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
p

Peter Lim

12/08/2022, 10:57 PM
Hi, are there any examples on how to get custom run metadata from dbt runs? I want to get the row count of the model after its materialization (and plot it in the UI)
o

owen

12/08/2022, 10:58 PM
hi @Peter Lim!
runtime_metadata_fn
(https://docs.dagster.io/_apidocs/libraries/dagster-dbt#assets-dbt-core) sounds like what you're looking for here
p

Peter Lim

12/08/2022, 11:00 PM
thank you for the reply, Owen! Do you have an example function for this?
I was actually looking at this earlier but don’t know how to use it
o

owen

12/08/2022, 11:05 PM
yeah it's got a pretty nasty type signature for something with no examples 😬 . Easiest way to think about it is that it's a function with two arguments. The first is a
context
argument (that's the same thing that's passed into the
context
argument of an asset), and the second is a dictionary of metadata parsed out of the
manifest.json
file. So one example would be
Copy code
def custom_metadata_fn(context, node_info):
    table_name = node_info["name"]
    table_schema = node_info["schema"]
    n_rows = ... # query database
    return {"n_rows": n_rows}
then you can just have
load_assets_from_dbt_manifest(..., runtime_metadata_fn=custom_metadata_fn)
p

Peter Lim

12/08/2022, 11:08 PM
thank you! I will try this out tomorrow 🙂
9 Views