I'm getting an intermittant error when dagit is st...
# announcements
p
I'm getting an intermittant error when dagit is starting:
Object IPCErrorMessage(serializable_error_info=SerializableErrorInfo(message="dagster.check.CheckError: Invariant failed. Description: Can only serialize whitelisted namedtuples, received tuple ('2018-01-01', None)
but I'm having a lot of trouble finding the source of the Tuple. Thoughts on where this would be coming from?
Full output^^
a
hmm any chance that date is hardcoded anywhere? A rogue
,
in yaml or python could be turning something in to a tuple on accident
p
There are a few places where the date is hardcoded, but I've systematically changed all of them and the date in the error hasn't changed. I'm afraid it's something to do with our kubernetes setup
a
umm double check that you are using the image&tag you expect and are not on a stale one by accident?
image pull policy may be a thing too, if you are re-pushing the same image tag
p
It had to do with docker image reuse: forcing a new image seems to have fixed it
👍 1