Hi all, I've mostly been using

return Output(value, metadata={...})

for my assets and I'm very happy with that. Now, I wanted to use the same for a dynamic graph. I cannot use the same pattern or I get an

AttributeError: 'InvokedNodeOutputHandle' object has no attribute

because I'm expecting it to be a pandas DataFrame but it's not. The result is the output of running

pd.concat

on the collected dynamic value. Any ideas?

I'm also using a dynamic output

-> DynamicOutput[dict]

and returning a

dict

with all the information needed. I will try your suggestion of adding metadata as part of

Output

Yes, I don't think it's a solution to your problem. I think you need to configure your I/O manager.

Hi Moritz. Not sure how you're running into the error above, but the following worked locally for me:

@op(out=DynamicOut())
def load_pieces():
    df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
    large_data = [df, df, df]
    for idx, piece in enumerate(large_data):
        yield DynamicOutput(piece, mapping_key=str(idx), metadata={"idx": idx})


@op
def concat_pieces(context, pieces: List[pd.DataFrame]):
    <http://context.log.info|context.log.info>(pd.concat(pieces))


@job
def dynamic_graph():
    pieces = load_pieces()
    concat_pieces(pieces.collect())