:dagsir: Tell us about how you organize your Dagster code
Hi everyone, I’m looking to learn more about how you organize your complex Dagster projects and the troubles you are facing when organizing the code. Besides, when structuring your projects, are there any goals you’re trying to optimize?
We’re working on a more comprehensive recommendation for Dagster project structure. In the past, we’ve come up with the experimental Create a New Project which comes with a CLI command that scaffolds a project skeleton, but we’ve heard great feedback about it not being well suited for larger complex and prod-scale projects. So I’d love to learn from you about any good practices when you design and structure your data projects.
Here’s a GitHub discussion for collecting feedback and scenarios.
07/20/2022, 5:22 PM
do you want feedback here or is there a github discussion?
will be following this closely 👀
we’re looking into Dagster as a potential replacement for AWS Glue jobs+workflows and we have a lot of those, so very interested in best practices around organizing complex projects.