have anyone tried this with Dagster? https://prql-lang.org/ dbt is powerful and we use it heavily now. With Dagster, I could see a future without the need of dbt DAG. one other benefit of dbt is the SQL + jinja, just curious if there is a nice way to write those super-charged SQL transformation easily in Dagster without dbt.

Nothing is preventing you from using Jinja w/o dbt :) that being said prql does look quite cool! Lots of tools are coming up to enable lots of plug and play b/w databases and compute engines I'm quite excited for it.

you might enjoy reading through this - https://github.com/stkbailey/dagster-jaffle-shop

yeah @Hui Zheng, the real interesting thing if you go down this path is that you realize how much dbt handles for you in the development process for you, since it just looks at the SQL files. But you can absolutely create a similar system, where your Dagster job could look at a set of PRQL files, or whatever, and compiles them at runtime, and you rarely have to change the actual Dagster Python, and instead just work with the YAML. Honestly it blows my mind all the cool pseudo-products you could build with Dagster-as-a-platform. I am personally waiting for someone to port all the Datahub scrapers into Dagster.

Thank you for all the great input. I will do more research. appreciate it!