This is certainly possible!
Different from what you outlined, but these two repos show an example of having dagster and dbt separate, but then combining them during dagster deployment into a single image
https://github.com/slopp/dbtproj
https://github.com/slopp/dagsterproj
The benefit of combining them is it allows you to use the asset abstraction instead of having to rely the op/job abstraction plus a custom wrapper that invokes dbt run in a separate image