i have a solid that does the download and works great when i configure the key as an input in YAML. i'm hoping to keep this as a standalone solid and then reference it once for each dataset

i tried to do this from the pipeline, but that doesn't seem like the right approach. next i tried creating a composite solid and reading the keys from YAML and aliasing the download solid for each key, but it looks like composite solids don't support context and therefore i'm not sure how to configure it

i see there is a mechanism for creating configurations on the fly which i haven't tried yet

it also seems like another possibility would be running a pipeline for each key and having another pipeline run after to do whatever merging/cleaning needed to happen

anyway, before i go too far down any other of these other paths i thought i'd see if there was any guidance from the group here...w

what's the best way to dynamically run the same solid from a list (ideally within the same pipeline)?

thanks in advance!

If so, you might find this example helpful: https://docs.dagster.io/examples/fan_in_pipeline