Hello All, If I am using map collect with an op that has three outputs I would like to pipe to 3 different functions. Looks like the example image. Has anyone had experience trying this successfully? I feel like I am close.

Hi Alexander. It’s not 100% clear to me the structure of `extract_telemetry_data`… Is it the op that is emitting the

DynamicOutput

? If it’s just operating on the mapped data, you might try defining a

graph

to break things up:

@graph(ins={"uuid": In(Any)})
def process_uuid_partition(uuid):
    raw, hourly, daily = extract_telemetry_data(uuid)
    load_telemetry_raw(raw)
    load_telemetry_hourly(hourly)
    load_telemetry_daily(daily)

I don't believe we currently support collecting across multiple mapped datasets. If collecting across those three functions is important to you, I'd suggest essentially collapsing them into a single op and having

extract_telemetry_data

yield three times as many outputs (each one for raw, hourly, and daily)

Like @sandy said, I couldn't collect across multiple sets, but if the mapped op returns a tuple you can operate on each of the n-tuple output independently in separate maps or collects if you unpack them which is incredibly cool... But here is a working example now as an FYI. For the context @prha, The get_unit_uuids is returning the dynamic output, extract_telemetry_data is returning a three-tuple, I needed to map the tuples independently. Thanks for your insight as well.