I’m new to Dagster, and this question is on how I should structure my code
I’m extraction data from an API with more that 300 endpoints
The for extract, transform (convert from xml to a table), load (csv an an S3 bucket)
The way I understand Dagster is that each extracted endpoint, converted xml and csv in S3 is an
asset
.
How do I avoid writing +900 assets?
c
chris
01/17/2023, 10:40 PM
you can view each of these as partitions of an asset (partitioned based on the endpoint)
m
Martin Carlsson
01/19/2023, 6:22 AM
@chris Thanks
Yea maybe! I’ll read the documentation