https://dagster.io/ logo
#ask-community
Title
# ask-community
o

Olivier Dupuis

04/07/2022, 3:09 PM
Good morning everyone! I’m sorry if this going to be badly formulated question, but I’m still quite new to the Dagster world. I have an op for which I’d like to materialize multiple assets. I’ve been able to yield a single asset with an op, but I can’t figure out how to materialize multiple assets and yield them all in a single call to that op. Am I trying to do something that cannot be achieved? I tried searching the documentation and Slack archives and can’t find a similar example. Here’s the op I’m working with…
Copy code
@op(
    required_resource_keys = {"novacene_client"}
)
def store_ml_enrichment_files(context, df_ml_enrichment_files):
    s3 = boto3.resource('s3')

    for index, row in df_ml_enrichment_files.iterrows():
        # Read csv as pandas
        df_ml_enrichment_file = context.resources.novacene_client.get_file(row['file_path'])

        # Extract date from file name
        file_date = row['name'].split("_")[2].split(".")[0][0 : 8]

        # Save df as csv in S3
        csv_buffer = StringIO()
        df_ml_enrichment_file.to_csv(csv_buffer, index = False)
        s3.Object('discursus-io', 'sources/ml/' + file_date + '/ml_enriched_' + row['name']).put(Body=csv_buffer.getvalue())

        # Materialize and yield asset?
    
    return None
Thanks for your help! 🙂
1
z

Zach

04/07/2022, 3:53 PM
what kind of issue are you having so far? I've never had any problems yielding multiple assets from an op by just doing something like
Copy code
...
yield AssetMaterialization("materialization1")
yield AssetMaterialization("materialization2")
...
or even
Copy code
for i in range(5):
    yield AssetMaterialization(f"materialization_{i})
o

Olivier Dupuis

04/07/2022, 4:09 PM
Well that’s embarrassing. I don’t know what I was doing wrong previously, but your approach definitely works. Thanks @Zach 👍
z

Zach

04/07/2022, 4:10 PM
No worries, that's programming! Dagster on! dagster spin
🎉 1
2 Views