https://dagster.io/ logo
#ask-ai
Title
# ask-ai
m

Mikael Frosthage

02/09/2024, 8:37 AM
How do I log a panda dataframe as a nice table, instead of text in a log entry
s

Scout

02/09/2024, 8:37 AM
To log a Pandas DataFrame as a nicely formatted table in Dagster, you can use the
TableMetadataValue
class to create a metadata entry that renders the DataFrame as a table in the Dagster UI. However, it's important to note that as of the last update from the documentation, there might be limitations on the data types supported by
TableMetadataValue
. Here's an example of how you might log a Pandas DataFrame using `TableMetadataValue`:
Copy code
python from dagster import op, Out, MetadataValue import pandas as pd  @op def my_op(context):     df = pd.DataFrame({         "fruit": ["apple", "banana", "cherry"],         "quantity": [5, 3, 8]     })          # Log the dataframe as a table     context.log_event(         MetadataValue.table(df.head().to_dict('records'), df.columns.tolist())     )          return df
In this example, the
to_dict('records')
method is used to convert the DataFrame into a list of records (dictionaries), which
TableMetadataValue
can then render as a table. The
df.columns.tolist()
provides the column names for the table. Please note that the exact method for logging a DataFrame as a table may vary depending on the version of Dagster you are using, and the capabilities of
TableMetadataValue
may have changed since the last documentation update. If you encounter any issues with data types not being supported, you may need to convert those columns to a supported type before logging, or check the latest Dagster documentation or GitHub issues for updates on this feature.
6 Views