Happy new year folks! :tada: Is there a way curre...
# ask-community
j
Happy new year folks! 🎉 Is there a way currently to automatically trigger a dbt run as soon as it’s upstream assets are materialized? I have a scenario where there are: • Multiple airbyte connection (with multiple assets produced per connection) • Multiple dbt projects that consume from airbyte assets I want a sensor to automatically trigger a dbt project as soon as it’s upstream assets have all been materialized.
🤖 1
a
Happy New Year! 1. Create multiple jobs, one for each set of dbt models you want to trigger. This could be an entire project, or even a subset of a project. (We have 20+ jobs using the same project, but different models or same parameterized models with different vars) 2. Create multi-asset sensors to trigger those jobs, based on the appropriate asset selection You can also create a combined asset job for airbyte + dbt, without sensors. We use sensors for triggering dbt tests/doc generation, but the model runs are handled as a combined asset job with airbyte assets (dagster will handle the dependencies, as long as they are correctly represented in your asset graph).
j
They show up as different asset keys. The one defined as a dbt source appears as
postgres/mart_gh_stargazer
, and the one defined as a model in project 1 appears as
posgres1/mart/mart_gh_stargazer
a
You’ll need to get the to be the same asset key. (There may be other ways, but that’s what I’m familiar with.) A few things that might help you with that are the
source_key_prefix
and
node_info_to_asset_key
arguments to https://docs.dagster.io/_apidocs/libraries/dagster-dbt#dagster_dbt.load_assets_from_dbt_manifest I suspect the latter is going to be what you want to modify. The default mapping function is defined here for reference: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-dbt/dagster_dbt/asset_defs.py#L133