https://dagster.io/ logo
Join Slack
Powered by
Hello, I am getting this error when using celery_k...
# ask-community
a
Hello, I am getting this error when using celery_k8s_job_executor for executing Jupyter notebooks as an op using Dagstermill op definition. 
dagster._core.errors.DagsterSubprocessError: During celery execution errors occurred in workers:
[test_dict]: dagster._serdes.errors.DeserializationError: Deserialized object was not expected type <class 'dagster._core.events.DagsterEvent'>, got <class 'dict'>
Error Screenshot
j
Are you by chance on a team with https://dagster.slack.com/archives/C01U954MEER/p1679487314949989 or did you guys run in to this separately
I’m curious about the name 
test_dict_job
, is that just you trying to debug this
What dagster version is this?
Ah I believe I have a fix for this. I suspect your step is writing to stdout with a line that parses as json? E.g. just 
{}
, or a whole object? Celery-k8s does some hacky parsing of stdout from steps that might be picking that up. Putting up a fix
👍 1
m
@johann Yes! we are a team. Currently, we are running on dagster 1.2.2 on EKS
@johann Is https://github.com/dagster-io/dagster/pull/13143 the fix you are talking about? If yes! Will this be included in next release?
j
That’s the planned fix yep. Likely to be in the release next week. Would you be able to confirm
I suspect your step is writing to stdout with a line that parses as json? E.g. just 
{}
, or a whole object?
a
its a very simple notebook that we are executing. Cell1 --> declare 2 variables a=1 b=2 Cell2 --> print above 2 variables print(a,b) that is the complete notebook
j
The fix went out in 1.2.4 yesterday
a
thanks @johann I will test and confirm
8 Views
Open in Slack
PreviousNext
Powered by