Hello, is anyone using Great Expectations with Dag...
# ask-community
j
Hello, is anyone using Great Expectations with Dagster, and successful at exporting the validation results to a human readable format? I'm using the
ge_validation_op_factory_v3
in my pipeline, from which I can get this expectation result markdown on the dagit UI and also separately export the results json. But then I'd wish to extract the markdown/some human readable format
c
You should be able to extract the markdown from the yielded ExpectationResult object - you should be able to do something similar to here to retrieve the expectation result event log entry; and pull the metadata off of it:
j
Thanks @chris, got this to work with this function which I'll rework if possible
Copy code
@op
def get_gx_results(context, df, expectation):
    """
    Standard op that exports test results to a HTML file
    :param context - dagster context
    :param df - dataframe for testing
    :param expectation - the GX expectations op

    Returns
    df for next step processing
    """
    
    # get Dagster event results
    instance = context.instance
    # get expectation op results
    log_entry = instance.get_event_records(EventRecordsFilter(event_type=DagsterEventType.STEP_EXPECTATION_RESULT))[0]
    
    dagster_data = log_entry.event_log_entry.dagster_event.event_specific_data
    expectations_metadata = dagster_data.expectation_result.metadata_entries
    metadata_str = expectations_metadata[0].value[-1]

    # get filename for export
    filename = f'great_expectations_results/feature_test/{expectation_name}_{test_time}.html'
    with open(filename, 'w') as w:
         w.write(metadata_str)
   
    # # pass dataframe to next stage
    return df