Hello! I am trying out dagster and have the following use case: Retrieve an arbitrary number of files from an external service (around 800, but not fixed), process each of those individually, then merge them and perform further processing. How would I approach this in dagster? As far as I understand there isn’t a way to define arbitrary outputs of an asset

Don’t know if this works or not, but you could try using a dynamic op: https://docs.dagster.io/concepts/ops-jobs-graphs/dynamic-graphs#a-dynamic-job and only return the result from

foo(bar.collect())

as an asset

if you're trying to do this with software-defined assets, here's the best that we've got right now: https://github.com/dagster-io/dagster/issues/9559#issuecomment-1270755525