Custom IO managers and metadata: Would be nice to be able to read the metadata produced by

handle_output()

in the

load_input()

method. In my attempts, using

add_output_metadata()

does work as expected – the event data in asset catalog displays the specified keys – but the keys are not available when reading

upstream_ouput

property of the context in

load_input()

. Modified example from the doc:

class DataframeTableIOManagerWithMetadata(IOManager):
    def handle_output(self, context, obj):
        table_name = context.name
        write_dataframe_to_table(name=table_name, dataframe=obj)

        context.add_output_metadata({"num_rows": len(obj), "table_name": table_name})

    def load_input(self, context):
        table_name = context.upstream_output.name
        
        num_rows = context.upstream_output.metadata["num_rows"]  # <-- That'd be great 

        return read_dataframe_from_table(name=table_name)

If not using the approach suggested above, how else can I read the metadata in the

load_input()

method?

Maybe this will help?

Yes, we’re tracking this in that issue-- hope to have it solved in the next few releases.