Using `build asset reconciliation sensor` and get this error

Question

Using `build asset reconciliation sensor` and get this error ```dagster check CheckError Failure condition File usr local lib python3 10 dist packages dagster core errors py line 206 in user code error boundary yield File usr local lib python3 10 dist packages dagster grpc impl py line 328 in get external sensor execution return sensor def evaluate tick sensor context File usr local lib python3 10 dist packages dagster core definitions sensor definition py line 428 in evaluate tick result = list self evaluation fn context File usr local lib python3 10 dist packages dagster core definitions sensor definition py line 593 in wrapped fn result = fn context File usr local lib python3 10 dist packages dagster core definitions asset reconciliation sensor py line 954 in sensor run requests updated cursor = reconcile File usr local lib python3 10 dist packages dagster core definitions asset reconciliation sensor py line 769 in reconcile = determine asset partitions to reconcile File usr local lib python3 10 dist packages dagster core definitions asset reconciliation sensor py line 393 in determine asset partitions to reconcile stale candidates latest storage id = find parent materialized asset partitions File usr local lib python3 10 dist packages dagster core definitions asset reconciliation sensor py line 244 in find parent materialized asset partitions if child asset key in target asset keys and not instance queryer is asset in run File usr local lib python3 10 dist packages dagster utils caching instance queryer py line 148 in is asset in run check failed File usr local lib python3 10 dist packages dagster check init py line 1699 in failed raise CheckError f Failure condition desc ``` The three assets directly referenced with it are time partitioned daily monthly monthly There are downstream assets not directly referenced Any idea on the issue

owen · Answer

hi < Alec Koumjian> what version of dagster are you on currently This stack trace seems to indicate an earlier version I m not confident if an update would resolve the issue but I wouldn t be surprised Just for some more info did you happen to delete any runs recently before this error showed up

Alec Koumjian · Answer

Yeah this is on `1 1 21` I did delete some runs but I can t remember now I had a large set of assets in here before and I m been slimming it down to try to reduce the complexity of what s being reconciled I ll deploy recent version to my test instance and see what happens

Alec Koumjian · Answer

Most recent version `1 2 3` appears to produce an identical result Worth noting the list of Jobs seems incorrect referencing all sorts of downstream and upstream assets Is it possible that the definition is not being updated after changing the list of assets defined in the sensor job

Alec Koumjian · Answer

I am able to produce the failure with specifying just two assets The first has the following asset definition ``` asset partitions def=DailyPartitionsDefinition start date= 2018 06 01 code version= 0 0 2 retry policy=RetryPolicy max retries=6 delay=10 0 seconds backoff=Backoff EXPONENTIAL 10 20 40 80 160s 320s of delay jitter=Jitter PLUS MINUS freshness policy=FreshnessPolicy maximum lag minutes=5 def extract method ``` The second has this definition ``` asset partitions def=MonthlyPartitionsDefinition start date= 2018 06 01 code version= 1 ins= extract method AssetIn metadata= allow missing partitions True def transform method context extract method gt Output DataFrame ADAMETLSourceSchema ``` If I only include the extract asset it s content Adding the transform asset I get the above error I tried removing the `allow missing partitions` and it still didn t resolve Is it possible it is failing simply because the transform step is mapping monthly partitions onto daily extract partitions

Alec Koumjian · Answer

The daily gt monthly mapping also does not appear to be the source of the issue

Alec Koumjian · Answer

Okay it looks like the error occured simply because there was more than a single missing partition in the extract asset I can maybe understand not queueing up all missing partitions automatically but I didn t get any kind of helpful error message

owen · Answer

hi < Alec Koumjian> do you mind sharing the stack trace that you re seeing in `1 2 3` I m haven t been able to replicate this

Alec Koumjian · Answer

I attempt to go back to that implementation and now can no longer replicate it I am having some issues with sensors in general but I or my colleague may bring that up separately