The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi, my assets lineage blends dbt, athena and python assets. This results in the dbt models being grouped in distinct `dbt_run_` ops at run time. Unfortunately some of their dependencies are lost in the process, and they are built in the wrong order. I put together a <https://www.loom.com/share/5d453d3988bb43bc978e2e3ad68f48ab|Loom> and a quick follow-up <https://www.loom.com/share/22485765a61649c09be980a19ba741aa|Loom> that highlight the problem.

hi <@U01AV4K3W14>, you were just helping me on <https://github.com/dagster-io/dagster/discussions/8873|discussion #8873>, this is the same pipeline. Maybe you know what's going on?

This problem is still happening for me on 1.1.19.
<@U01AV4K3W14> or maybe <@U01J51Y6B9D> do you have any ideas?

Hey Remi, sorry for the late reply. Do you mind filing an GitHub issue regarding this problem so that we can track it? This looks like a problem with our asset subsetting algorithm - we use that to determine the order in which assets are materialized as well as to ensure that there are no cycles when dependencies are defined.

Hi <@U01N6UG0JA3> <@U01AV4K3W14> - can either of you point me to the GitHub issue that was filed?

hi Sam, the bug disappeared so I did not end up filing an issue actually

I just ran this issue again so I filed it <https://github.com/dagster-io/dagster/issues/13437|here>