Monorepos 4 lyfe. I thought about this kind of thing for a long time and I wanted to do multi-repo, but came to the conclusion that if multiple repos share the same code and have to coordinate on changes to that shared code, you basically just have a distributed monorepo and keeping things in sync will require a lot of extra coordination.
Alternatively, the Earthly build tool can trigger builds contained in external repos. Submodules can be kinda yucky, though they might work for you.