https://dagster.io/ logo
#dagster-plus
Title
# dagster-plus
a

Aman Gottumukkala

02/11/2024, 9:20 PM
What are the pros and cons of 1. running 5 independent asset schedules without any dependencies that materialize assets based on a postgres table that acts as a queue that independently creates new data vs. 2. Creating a traditional DAG pipeline with asset dependencies For context I'm creating a proprietary enrichment database of companies and each asset is fetching data about a company from a different datasource. I don't really see why I would create a traditional DAG pipeline when I can partition and manage the load of each asset independently which I don't think i can do with a traditional DAG pipeline.
j

johann

02/12/2024, 2:17 PM
If there’s no dependency between the assets then agreed it may make sense to schedule them separately
3 Views