hi, is there a way to create multiple assets from ...
# ask-community
p
hi, is there a way to create multiple assets from a single config? I want to pass a text in a config and split the text into sentences, then I want to generate a different downstream asset for every sentence, what is the best way to implement that?
z
Sounds like you're looking for an asset factory
p
emm I will refine the question, I need to materialize the assets dynamically at run time. not creating multiple definitions of an asset. may be I can use Partitions for it?
z
Yeah seems like a dynamically-partitioned asset could work. Or if you don't necessarily need to track each sentence as an individual asset you could use a graph-backed asset with a fan-out via a dynamic graph. That's probably how I would do it.
p
yeah, but I want to create a downstream assets with the sentences, and I just can figure out how to do it, basically if I have a text=(s1, s2,s3) I want to create 3 image generation prompts (p1,p2,p3) and then 3 images (im1, im2, im3) and then aggregate them into a single context, so: text => (s1, s2, s3) => (p1, p2, p3) => (im1, im2, im3) => text_images. and I am not sure partitions support that kind of operations.
z
Sounds like a dynamic graph-backed asset would be much more straightforward. You could fan-out across your sentences in your first step, chain the image generation and image steps, then aggregate them in an op that handles the collected results of the chained fan-out. Alternatively you might be able to chain dynamically-partitioned assets with partition mappings. You can't really dynamically create assets at runtime as Dagster needs the assets to be defined before a graph is constructed
p
ok, got it. thanks