Folks, is there any timeline on <https://github.co...
# dagster-feedback
j
Folks, is there any timeline on https://github.com/dagster-io/dagster/issues/4364 ? Having nested dynamic fan-out would be extremely helpful for some pipelines my team is authoring. Right now, we have to: 1. Generate an asset per top-level dynamic output. 2. Sleep ~60s between asset generation so we don't get burned by this issue This seems kinda hacky to achieve behavior that would normally exist in an ETL system. Additionally, if we have multiple schedules for this run, this mechanism breaks down b/c we'd skip runs. Looking for some support here 🙏 .
j
@alex do you know if there are plans to work on #1? for the second issue, you can use the new
multi_asset_sensor
to work through a list of multiple asset materializations. something like this
Copy code
@multi_asset_sensor(
    asset_keys=[AssetKey("asset_a")],
    job=log_asset_sensor_job,
)
def asset_a_sensor(context):
    all_asset_a_events = context.materialization_records_for_key(
        asset_key=AssetKey("asset_a"), limit=5
    ) # you can set limit to whatever batch size you want

    for event in all_asset_a_events:
        context.advance_cursor({AssetKey("asset_a"): event})
        return RunRequest(
            run_key=context.cursor,
            run_config=...,
        )
a
I’m not aware of plans for when #4364 will get worked on