Hello :wave: I'm looking for recommandations regar...
# ask-community
f
Hello đź‘‹ I'm looking for recommandations regarding structuring dagster pipelines in monorepo vs. multirepo. There is not many information online, besides this discussion (https://github.com/dagster-io/dagster/discussions/8972)
s
Thoughts here @yuhan?
y
Hey @Félix Tremblay assuming you’re talking about putting pipelines in a single Dagster repository vs multiple Dagster repositories. I view the benefits of separating your pipelines into multiple Dagster repositories are: 1. Organizational purpose: different teams/orgs can arrange their jobs/sensors/schedules/assets in their own way (i.e. in their own Dagster repository) while keeping a single Dagster deployment — a deployment can have multiple Dagster repositories. a. However, if you’re using Software-defined assets, you can get this benefit also out of separating assets multiple `AssetGroup`s (managed by different teams) within a single repository. 2. Deployment and environment separation: Multiple Dagster repositories can be helpful when you want to keep conflicting Python dependencies separate, or keep your jobs in different Python environments. Also, when one repository (say managed by teamA) fails to load, teamB’s repository won’t be affected. So, the main benefit of multirepo is being able to separate environments. If you don’t see the need to do so, I’d recommend keeping things within in one repo to start with + manage semantic groups using asset groups.
f
Hello @yuhan, Thank you very much for this detailed and comprehensive answer. It helped me a lot and clarified the options and ambiguities.