The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

I'm seeing my asset reconciliation sensor creating duplicate runs for the same asset. I'm thinking that while the initial run is going, the upstream asset completes and then while the downstream asset op is initializing, the sensor ticks again and creates another reconciliation run for that asset. Does that sound plausible?

The workaround in my mind is that I should increase the duration between sensor ticks

created using `load_assets_from_dbt_manifest`

<@U02N9H9MM33> It's definitely possible, although the sensor should take into account in-progress runs when determining if it will kick off a new run. Are you using freshness policies here? if so, are they on the dbt assets, or the downstream asset(s) (or both)?

think I have a good example where the log timestamps indicate that the latter run started before the initial run

the `load_assets_from_dbt_manifest` assets don't have some kind of magic run launcher right? they still rely on something like a sensor to detect that they're stale and initiate a run?

ok so I'm not duplicating them with my sensor at least

hey just checking back in on this! were you able to find those logs?

0b14301f-be93-4c3c-a4b9-91150f8d0d9d.gz

4852c503-68bc-4cf6-94f0-923ae87d2a00.gz

Hey <@U01J51Y6B9D>! thanks for following up. I was out the past couple days. Here are the logs for 2 runs of the same asset and partition from the same sensor

the asset key in question is `kippnewark / dbt / powerschool / stg_powerschool__assignmentscore`

run 0b14301f is enqueued at 10:11:36 for both upstream and downstream assets

run 4852c503 is enqueued at 10:14:19 for the duplicate run

that asset's step worker starts at 10:13:53 in the original run but execution doesn't start until 10:14:19

I thought they might be just duplicate logs looking at the timestamps but they're running in different pods

my thought is that any checks are looking only for running materializations whereas it should be looking for enqueued materializations

<@U011CET83FG> here's the details for the issue I mentioned in our DM

ah <@U02N9H9MM33>, I looked into this and you're 100% correct on this. We do not factor in runs that are in the QUEUED state when determining the state of a given asset. There is a bit of risk here, as runs can sometimes get stuck in the QUEUED state, but I think we can filter out old runs manually, and this should be a fairly simple fix. Thanks for the report :slightly_smiling_face:

just merged in a fix for this, it'll be in next week's release