https://dagster.io/ logo
#ask-community
Title
# ask-community
f

Félix Tremblay

04/26/2023, 3:09 AM
Hello 👋, I can't find how we can link the source tables of a dbt project to assets that are created in ops using AssetMaterialization events. Since the asset names are known at run-time, does it mean that the assets can't be shown in the DAGs in the UI? Can Dagster use the materialization events in order to know that some upstream dependency of the DBT project has been updated? Thanks!
v

Vinnie

04/26/2023, 8:52 AM
What I did on my setup was a custom function under
node_info_to_asset_key
to map my assets to dbt sources. It works since I’m not using `op`s, but have my entire platform based around assets. I’m not 100% sure on this, but I’d assume you can still map the materialization events to the dbt sources by passing the right key to the
AssetMaterialization
event, i.e.
AssetMaterialization(asset_key=AssetKey(["my", "dbt", "source", "key"]))
👍 1
s

sandy

04/26/2023, 5:13 PM
Hey Felix - if you have a dbt source named "asset0" and a model named "asset1" that depends on it, you should see something like this in your asset graph:
👍 1
if you click on "asset0", it will show you all materialization events for "asset0", even if they came from an op (not an asset)
❤️ 1
is there something more that you're looking for?
f

Félix Tremblay

04/27/2023, 12:03 AM
Thanks @sandy
I'm also wondering if that's taken into account for declarative scheduling? In other words, will asset1 become "stale" when there are new materialization events for asset0?
s

sandy

05/02/2023, 12:52 AM
I just tried this out, and it appears that the answer is no. Out of curiosity, why use ops + AssetMaterialization events instead of assets?
f

Félix Tremblay

05/06/2023, 3:23 AM
Hey @sandy, thanks for the answer! To address your question, ops + AssetMaterialization events felt more natural than assets for our particular use case. It's a bit hard to explain succinctly, but the main reason is that the tables are loaded incrementally and in many small chunks with dynamically optimized chunk sizes at runtime
2 Views