https://dagster.io/ logo
#ask-community
Title
# ask-community
l

Lawrence Shaban

08/23/2023, 5:21 PM
Hi everyone, I am having trouble with the dbt integration. I have successfully loaded in my entire dbt project but I want to segment it into multiple data pipelines for scheduled running in order to save on compute. It seems like using tags is the easiest way to achieve this?! But I haven't got it working. I feel like from the documentation the code I'm supposed to use is either "DbtManifestAssetSelection" or "build_dbt_asset_selection" but haven't managed to successfully implement either! Any help would be much appreciated! Thank you! https://docs.dagster.io/_apidocs/libraries/dagster-dbt
r

rex

08/23/2023, 5:43 PM
#dagster-dbt is a better channel for these questions. It would also be helpful it see: • What code you are using • What you are seeing • What you expect to happen We have documentation on this here as well: https://docs.dagster.io/integrations/dbt/reference#scheduling-jobs-that-contain-only-dbt-assets
l

Lawrence Shaban

08/23/2023, 7:27 PM
Hi @rex, apologies I didn't realize there was a dedicated #dagster-dbt channel! Thank you for answering though, it is appreciated! I've attached a screenshot of the code I was running, it wasn't complete so I wasn't really expecting anything bar it not erroring. The end goal was to have an asset graph of only the tagged dbt models with upstream and downstream dagster assets and to have them run on a specific schedule. It is only the "tag_selection" line that causes the issue, otherwise dagster runs. This line gives the error "get_manifest_and_translator_from_dbt_assets metadata_by_key = dbt_assets_def.metadata_by_key or {}". Thanks again for the help!
r

rex

08/23/2023, 7:34 PM
you’re not passing the correct reference there:
Copy code
tag_selection = build_dbt_asset_selection([app_prod_pipeline_dbt_assets], dbt_select="tag:crf")
app_prod_pipeline_dbt_assets
not
dbt_assets
🌈 1
l

Lawrence Shaban

08/23/2023, 8:54 PM
Thank you! 🙂
d

David Weber

08/24/2023, 8:34 AM
Hi @rex, in this case there is an error in the API reference: https://docs.dagster.io/_apidocs/libraries/dagster-dbt#dagster_dbt.build_dbt_asset_selection Here the example says:
Copy code
@dbt_assets(manifest=...)
def all_dbt_assets():
    ...

# Select the dbt assets that have the tag "foo".
foo_selection = build_dbt_asset_selection([dbt_assets], dbt_select="tag:foo")
Which then should be:
Copy code
# Select the dbt assets that have the tag "foo".
foo_selection = build_dbt_asset_selection([all_dbt_assets], dbt_select="tag:foo")
👍 1
16 Views