https://dagster.io/ logo
Title
m

Martin Carlsson

01/13/2023, 4:19 AM
I’m new to Dagster, and this question is on how I should structure my code I’m extraction data from an API with more that 300 endpoints The for extract, transform (convert from xml to a table), load (csv an an S3 bucket) The way I understand Dagster is that each extracted endpoint, converted xml and csv in S3 is an
asset
. How do I avoid writing +900 assets?
c

chris

01/17/2023, 10:40 PM
you can view each of these as partitions of an asset (partitioned based on the endpoint)
m

Martin Carlsson

01/19/2023, 6:22 AM
@chris Thanks Yea maybe! I’ll read the documentation