Hi again Dagsters! Another question for you. Is it possible to programmatically instantiate solids - for example, if

solid_a

outputs a list of

n

files, and we want to create

n

aliases of solid_b each with a file as input, can we do this?

You can do this at pipeline creation time with the "composition functions" using

@pipeline

or

@composite_solid

if you know about the

n

files ahead of time. If the

n

files changes every-time this is approach is basically creating a different custom pipeline every time. If you want to do this at pipeline run time over a dynamic set of

n

files - we don't have great tools for this yet (but something we are looking to do in the future). You can create a fixed width set of `solid_b`and then dynamically divide the

n

files amongst them. An example of this can be seen here https://github.com/dagster-io/dagster/blob/master/examples/dagster_examples/toys/sleepy.py

Thanks Alex. Might look into rewriting the solid itself to handle a variable length input, rather than calling it a variable number of times.

Good luck, let us know if you have any other questions 👍