https://dagster.io/ logo
#dagster-support
Title
# dagster-support
l

Lorenzo

07/25/2022, 1:15 PM
Hi everyone! I am working with dbt dbt + dagster dagster yay. I am importing my dbt models automatically using
load_assets_from_dbt_project
. Is it possible to distinguish each dbt model based on the tag that is associated to? For example: i tag only one model with the tag "finance" and in dagster I want to run only the specific model who has the "finance" tag, which could be only one, or even more than one.
🤖 1
🌈 1
s

sean

07/25/2022, 4:16 PM
cc @owen
o

owen

07/25/2022, 4:23 PM
hi @Lorenzo! Dagster doesn't currently have a concept that's equivalent to dbt tags to create a selection from, but there are a couple of options. The first is if you only ever want to run models with the finance tag using dbt, you could
load_assets_from_dbt_project(..., select='tag:finance')
. The select argument here takes in a dbt-syntax select statement, and will limit the models you load to only ones matching that statement. Another option, if you don't have overlapping tags (i.e. each model will only ever have at most a single tag) would be to map the tag of the model to dagster's "group" concept. You can do this by setting the
node_info_to_group_fn
(https://docs.dagster.io/_apidocs/libraries/dagster-dbt#dagster_dbt.load_assets_from_dbt_project) argument to something like
lambda info: info["tags"][0]
(didn't test that, but it'll be that basic shape). This way, each dbt asset loaded will be mapped to a group based on its first tag, and you can select from those groups using
AssetSelection.groups("finance")
we're also considering adding a concept that's more similar to tags, but this would be further down the line
l

Lorenzo

07/26/2022, 9:20 AM
Thank you so much for the explanation @owen! I am going to try this right now and I am looking forward to the possible "tags update" yay
j

James Hale

10/08/2022, 3:42 PM
@Lorenzo we solved this by taking control over how asset keys are generated for the dbt assets by providing a function to the
node_info_to_asset_key
parameter of
load_assets_from_dbt_project()
.
l

Lorenzo

01/13/2023, 10:00 AM
@James Hale Hi James, I see only now your message because I've been working on another project for a while. I looked into this
node_info_to_asset_key
but I can't understand how to use it 🥲. My idea was to import all the assets at once (with one statement, like this:
dbt_assets = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR)
, but still have the control over one single dbt model at a time, so that I could restart the dag "from point of failure" without re-materializing also the upstream models that succeded. How should I set up the "node_info_to_asset_key" to do so?
dbt_assets = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, node_info_to_asset_key= [ ???? ])
Just to give some context: at the moment I successfully reached my goal by just importing one model at a time, but the project is getting exponentially big and models are now hundreds 👀
Copy code
# Load the assets from your dbt project automatically using models names
dbt_assets_1 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="first_dbt")
dbt_assets_2 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="second_dbt")
dbt_assets_3 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="third_dbt")
dbt_assets_4 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="fourth_dbt")
dbt_assets_5 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="fifth_dbt")
dbt_assets_6 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="sixth_dbt")
dbt_assets_7 = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, select="seventh_dbt")
2 Views