Howdy, folks! Newbie here.
I’m crawling a basic paginated API with Dagster.
Now I’m wondering if it makes sense to make every page a separate asset, and if yes, then how to do it.
Basically I’d like to spawn new assets from another asset: e.g. fetch the total page count, then spawn an asset for each page in parallel, and once all of them have completed spawn another asset that is going to merge all the pages together.