https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Marco

04/15/2021, 12:10 PM
Hello, I am currently experiencing an issue related to a partition set now deleted. Still, the error persists, I assume because of the underlying state of the daemon, which is persisted after restart. How to solve? Thanks
This happens with 0.11.4. Note that I can still run backfills but the nominal partition date ends up differing from the actual partition (‘version date’) that feeds through the solids in the pipeline. There is no obvious relation between the partition date and the resulting version date.
a

alex

04/15/2021, 3:20 PM
@prha
p

prha

04/15/2021, 3:46 PM
@Marco thanks for the report! I’ll put in a change to recover from that better in today’s release. In the meantime, I think you can go to the backfills page on
/instance/backfills
and cancel the problematic backfill
I’m not sure what you mean by `but the nominal partition date ends up differing from the actual partition`… is this a separate issue you’re running into?
m

Marco

04/15/2021, 3:49 PM
As far as I can tell there is nothing pending that can be cancelled in the UI
Queued and in progress lists are empty
As for this: `but the nominal partition date ends up differing from the actual partition`… i am not sure if related or not - I have noticed this after I experienced the error (btw: I find puzzling that I can run backfills when I also see ‘backfill daemon not running’)
As far as I can tell there is nothing pending that can be cancelled in the UI: I take it back, there was a backfill to cancel actually. After cancelling/restarting I don’t have the error anymore
👍 1
The strange behaviour with differing ‘effective’ and ‘nominal’ partition is also gone now if I run the partition from scratch. If I hit ‘re-execute’ instead, I have the same issue as before. In this latter case ‘Run configuration’ shows the nominal partition in tags but the version date in the config is wrong
p

prha

04/15/2021, 4:07 PM
Can you double-check your partition set definition? If there’s a mismatch between the partition listed in the tags and the config that is produced, I suspect it’s something with your
run_config_fn_for_partition
function
You can test this just by going to the
Playground
tab of the pipeline, and selecting the partition in the selector
m

Marco

04/15/2021, 4:11 PM
In playground I have the correct behaviour (version date in config = partition date in tag). Same as I have now (after resolving the error) when running backfill from ‘partitions’ tab
Actually the problem persists when running a ‘full’ backfill- as opposed to a backfill consisting of a single partition. I suspect this might have to deal with the parallelisation of runs and the use of partials in my code
I am still battling with this. Problem manifests itself when executing a backfill job consisting of several partitions. In this case the ‘version_date’ ends up always being the first partition being executed, whereas the tag reports the nominal partition date. I attach the code below, hopefully you can spot where the problem originates from or suggests way to debug. At this point I am clueless!
3 Views