Nicolas Parot Alvarez
08/01/2023, 9:19 AMload_assets_from_dbt_
functions to use dbt build
by default instead of dbt run
is not very intuitive.
The argument is that it will be more natural for someone following the tutorials, but I don't really understand it.
Running seeds does make sense, otherwise models depending on seeds may not be able to materialize, but I don't see the logic behind running "tests" and "snapshots" from a function that is supposed to load DBT definitions into Dagster objects.
Was the goal to run seeds at loading time and snapshots/tests are just a benign negative side effect?
https://github.com/dagster-io/dagster/pull/15060
(Reflection triggered by laziness to edit our code to add use_build_command=False
.)rex
08/03/2023, 3:57 PMload_assets_from_dbt_XXX
in the first place. Otherwise, your downstream models wouldn’t be able to run in the first place.Nicolas Parot Alvarez
08/03/2023, 4:24 PMdbt_seed_op
on a daily schedule, which is a bit excessive, but very practical to not have to bother with it during the deployment of a new version with a seed update.
I think including seeds during the loading of assets makes sense as a step in a new deployment. But ideally there would be something in place to only run them if they were modified, maybe something like https://docs.dagster.io/concepts/assets/asset-auto-execution#auto-materialize-policies-and-data-versions could be used ?
Besides, the running of tests and snapshots doesn't seem relevant to loading assets into Dagster, at least not by default.