The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

is there a way for dbt to have knowledge of a daily partitioned source asset in Dagster? I would like to build an incremental model that only does stale partitions, so it doesn't need to reprocess the entire asset when a new day is processed in the source asset

You can use the arguments `partitions_def` and `partition_key_to_vars_fn` in the `load_assets_from_dbt_*` functions

what if you have a project that source multiple assets with different partition defs?

You can load the dbt assets separately with the different partition defs, here is an example that has hourly, daily, and some non-partitioned dbt stuff: <https://github.com/dagster-io/hooli-data-eng-pipelines/blob/master/hooli_data_eng/assets/dbt_assets.py>

I tried this, but for some reason all the partitions are using the wrong partition data? What did I do wrong here?

This is what the source data looks like for every partition

What is your nhl daily partition? What about the dbt model?

`nhl_daily_partition = DailyPartitionsDefinition(start_date="1990-08-01")`

```select
    game_id,
    team,
    goals,
    row_number() over (partition by game_id order by goals desc) as rank
  from {{source('nhl_ingestion', 'raw_game_data')}}
  {% if is_incremental() %}
  where partition_key = '{{ var('datetime_to_process') }}'
  {% endif %}```

ok that all looks good, what do you mean `all the partitions are using the wrong partition data`? I guess I don't see the error in the screenshot

every partition I select has the same information under `Source data`  and when I check the new table on my database, partitions that say they've been completed aren't there