https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
g

geoHeil

01/28/2023, 12:31 PM
Is there any example for dagster + dbt incremental loadas where the incrementals map to dagsters partitions? Does the reconciliation sensor work aut-magically in such a secenario? Or is the manual definition of partitioned asset jobs a requirement?
o

owen

01/31/2023, 5:50 PM
hi @geoHeil! the
load_assets_from...
functions support both a
partitions_def
and a
partition_keys_to_vars
function, which allows you to map between a partition key and some variables to pass into your incremental models. This does require that all models loaded with that command share the same partitioning, so you may need to load your dbt assets across multiple invocations of
load_assets_from...
if some assets are partitioned and others are non-partitioned
g

geoHeil

01/31/2023, 5:53 PM
But this means that the reconciliation should work magically when the individual dbt assets are loaded with a sufficient partitioning strategy?
Where would these partitions show up in dagit?
o

owen

01/31/2023, 5:54 PM
yep! the reconciliation sensor will treat the partitioned dbt assets the same as any other partitioned asset (and same for dagit)
g

geoHeil

01/31/2023, 5:54 PM
I.e. The partitions tab for adhoc/ reconciliation does not exist
@owen is there any possibility to only load XOR normal and incremental models by only tagging the incremental ones? I.e. load all except incremental tags?
but
_exclude_
sould be the right start. Do you have any example on how to do this for a tag
my_tag_incremental
?
3 Views