Hi @Sven Lito - That was a great resource! Thank you! Love the asset_factory approach and will probably use some version of this. thank you box
@sean - I basically want to have a producer asset group and a consumer asset group. The producer pulls from one data store and serializes to S3. I then want the consumer assets (which should be dependent on their respective source assets) to pull from S3 and create a one-for-one reproduction of those tables in a different data store.
Note: I know this is a basic migration right now but the idea is that many consumers would then be able to consume from a single producer, reducing upstream cycles.
I was initially looking towards multi-assets for the producer but, thinking on the asset_factory approach, I could perhaps create another factory that recurses over the asset keys of the first factory and produces assets with "destination" versions of the source asset keys.... At least, I think that would work..... 🤔
Open to suggestions here too...