https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Colo Carlos

08/11/2023, 10:42 AM
Hi! What should I be checking for this error? v0.10.7
Copy code
Operation name: SchedulesRootQuery

Message: Invariant failed. Description: Attempted to deserialize class "TickData" which is not in the whitelist.

Path: ["repositoryOrError","schedules",0,"scheduleState","ticks"]

Locations: [{"line":168,"column":3}]

Stack Trace:
  File "/root/.local/lib/python3.8/site-packages/graphql/execution/executor.py", line 452, in resolve_or_error
    return executor.execute(resolve_fn, source, info, **args)
  File "/root/.local/lib/python3.8/site-packages/graphql/execution/executors/sync.py", line 16, in execute
    return fn(*args, **kwargs)
  File "/root/.local/lib/python3.8/site-packages/dagster_graphql/schema/jobs.py", line 328, in resolve_ticks
    for tick in graphene_info.context.instance.get_job_ticks(
  File "/root/.local/lib/python3.8/site-packages/dagster/core/instance/__init__.py", line 1396, in get_job_ticks
    return self._schedule_storage.get_job_ticks(
  File "/root/.local/lib/python3.8/site-packages/dagster/core/storage/schedules/sql_schedule_storage.py", line 167, in get_job_ticks
    return list(
  File "/root/.local/lib/python3.8/site-packages/dagster/core/storage/schedules/sql_schedule_storage.py", line 168, in <lambda>
    map(lambda r: JobTick(r[0], deserialize_json_to_dagster_namedtuple(r[1])), rows)
  File "/root/.local/lib/python3.8/site-packages/dagster/serdes/serdes.py", line 204, in deserialize_json_to_dagster_namedtuple
    dagster_namedtuple = _deserialize_json_to_dagster_namedtuple(
  File "/root/.local/lib/python3.8/site-packages/dagster/serdes/serdes.py", line 215, in _deserialize_json_to_dagster_namedtuple
    return _unpack_value(seven.json.loads(json_str), whitelist_map=whitelist_map)
  File "/root/.local/lib/python3.8/site-packages/dagster/serdes/serdes.py", line 239, in _unpack_value
    check.invariant(
  File "/root/.local/lib/python3.8/site-packages/dagster/check/__init__.py", line 168, in invariant
    raise_with_traceback(
  File "/root/.local/lib/python3.8/site-packages/future/utils/__init__.py", line 449, in raise_with_traceback
    raise exc.with_traceback(traceback)
y

yuhan

08/11/2023, 9:46 PM
Which dagster webserver (aka dagit) version are you on?
it’s likely your DagsterInstance schema is behind the version of your dagster or dagit. you can find the migration guide here and this one is for 0.10.x:https://docs.dagster.io/migration#migrating-to-0100
c

Colo Carlos

08/14/2023, 2:28 AM
Hi @yuhan,
dagit==0.10.7
dagster-aws==0.10.7
dagster-k8s==0.10.7
dagster-pandas==0.10.7
dagster-postgres==0.10.7
dagster-slack==0.10.7
dagster==0.10.7
Hi @yuhan, for full context I upgraded from
0.10.7
to
0.12.15
with no issues. Then I upgraded from
0.12.15
to
0.13.19
. Then I tried going back to
0.10.7
and that's when I got the error I reported. The instance migrate command was run each time.
it seems like
dagster instance migrate
doesn't work backwards
j

johann

08/16/2023, 4:12 PM
Correct. Probably not much help after the fact but if there’s a chance you’ll downgrade, it’s best to take a DB snapshot that you can revert to. In this case the newer dagster versions serialized the
TickData
object that the older version can’t handle
Since that’s stored in the db now, you can either • go back to 0.13 • restore the db from a snapshot if you have it • manually remove the rows that were updated with serialized tick data. There likely are other objects that could have been serialized too, but it probably depends on your usage while you were on newer versions
Out of curiosity, what caused you to revert back to 0.10?
c

Colo Carlos

08/17/2023, 5:43 AM
Thanks @johann. I reverted because I encountered issues in 0.13 and didn't have the bandwidth to look into it. I'm trying again now and our docker instance crashes during our scheduled runs so it seems to either be a resource issue or incompatible schedule configuration
Looks like a resource issue. A colleague had to separate the dagit/daemon service from the main dagster service into different pods when migrating to 0.13/0.14 so might have to do the same