geoHeil
05/18/2022, 4:54 PMyield MetadataEntry.md(docstring_schema, "DataFrame columns")
https://docs.dagster.io/concepts/ops-jobs-graphs/op-events can nicely be used to manually materialize the data and then manually log the metadata entry (which supports the markdown type). However when SDA are used the return value is the dataset and context.add_output_metadata
can add (string) based metadata. JSON seems to be recognized automatically - but MD not. Trying to directly feed it: context.add_output_metadata({'df columns': MetadataEntry.md(docstring_schema, "Dataframe Columns")})
fails with: DagsterInvalidMetadata: Expected a metadata value, found an instance of MetadataEntry. Consider instead using a MetadataValue wrapper for the valueowen
05/18/2022, 9:16 PMcontext.add_output_metadata({"Dataframe Columns": MetadataValue.md(docstring_schema)})
(basically, using MetadataValue
instead of MetadataEntry
). The reason that MetadataEntry does not work is that this class contains both the name of the field as well as the value, but when you pass in a dictionary of metadata, the keys already represent the field names.geoHeil
05/18/2022, 9:18 PM