The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

I think I broke dagit for some specific dataset/metadata which was logged as an asset materialization: SyntaxError: JSON.parse: unexpected character at line 1 column 950 of the JSON data. This is the error message the browser tells me - but dagit completely vanishes /goes white (only for this specific asset though).

Dagit.png

The reason was: I was logging some JSON wich contained empty keys and empty values (accidentally)

Thanks for this report. It looks like you’ve got a JSON metadata entry somewhere that is invalid JSON-- we should be catching this server-side.

Do you have any idea what that JSON value might be

still it would be nice if dagit does not vanish

Yeah I’ll create an issue for this, we should catch this gracefully.

agreed - it would be nice if you could catch this

pleas also share it here so I can track it.

when you say empty keys/values do you mean undefined, null, or empty string?

but it would be cool if you would catch the more generic parsing failure as well.

The error you posted is a parse failure, but empty key/value pairs is valid JSON: `JSON.parse(JSON.stringify({ [""]: "" }))`

no the JSON overall was more complex let me give you a more specific example in a minute

```import pandas as pd

df = pd.DataFrame({'foo':[1,2,3], 'bar':[4,5,6]})
m_derived = df.dtypes.astype('str').rename('dtype').to_frame()
d_c = 'description'

m_derived.loc['foo', d_c] = 'Foo description'
# notice bar is missing accidentally
m_derived.loc['', d_c] = ''

display(m_derived)
column_details = m_derived.reset_index().rename(columns={'index':'column_name'}).to_dict(orient='records')
column_details

m_res = {

    'source_upstream':{
        'name':'my_dummy_name',
        'column_details': column_details,        
    },
}
m_res


{'source_upstream': {'name': 'my_dummy_name',
  'column_details': [{'column_name': 'foo',
    'dtype': 'int64',
    'description': 'Foo description'},
   {'column_name': 'bar', 'dtype': 'int64', 'description': nan},
   {'column_name': '', 'dtype': nan, 'description': ''}]}}```

Thanks-- pretty sure the issue here is the nan, not empty strings

IIRC python treats nan as valid JSON even though it technically isn’t

ok so it looks like we need to figure out how to handle NaNs in JSON values server-side

<@U018K0G2Y85> issue Handle NaN in JSON metadata values on server

Created issue at: <https://github.com/dagster-io/dagster/issues/8080>