https://dagster.io/ logo
Title
c

chrispc

08/31/2021, 4:18 PM
Hi team, I am traying to generate an asset with a sample of data. I am using EventMetadata.json but the data has timestamp values and I am getting
TypeError: Object of type Timestamp is not JSON serializable or TypeError: Object of type date is not JSON serializable
. I know that json.dump() has the parameter default (
default
 is a function applied to objects that aren't serializable. In this case it's 
str
, so it just converts everything it doesn't know to strings) but I can't use it. If I use json.dump before I got the data in the assets catalog ugly
y

Yassine Marzougui

08/31/2021, 5:47 PM
Hi Chris - as a temporary workaround, you can monkey-patch Dagster's JSON serialization and use orjson which handles date types natively. All you need to do is add this before the pipeline code:
import orjson
from dagster import seven

seven.json.dumps = lambda data:orjson.dumps(data, option=orjson.OPT_SORT_KEYS).decode("utf-8", "replace")
c

chrispc

08/31/2021, 6:14 PM
Hi @Yassine Marzougui Thank you so much. Do you know where the dagster json serialization happens? I was lookin in dagster\serdes\serdes.py line 186 but I am not sure how to patch it
y

Yassine Marzougui

08/31/2021, 6:19 PM
If I'm not mistaken, it happens here and here, which is imported from here
👍 1
c

chrispc

08/31/2021, 6:20 PM
Thank you so much @Yassine Marzougui
👍 1
y

Yassine Marzougui

08/31/2021, 6:20 PM
You're welcome!