Filipe Almeida
02/13/2023, 2:17 PMload_assets_from_dbt_project
and I notice can load seeds as an asset if I set use_build_command=True
.
use_build_command (bool): Flag indicating if you want to use `dbt build` as the core computation
for this asset, rather than `dbt run`.
Could you tell me a potential problem the disadvantage of trying to load seeds as assets like this? Will it be slower for example?Stephen Bailey
02/13/2023, 4:13 PMseed
, run
, snapshot
, and test
(I think). The biggest issue may bethat you don't want to run test
every time you materialize an asset.owen
02/13/2023, 11:02 PMexclude="resource_type:test"
to load_assets_from...
to avoid that sort of thing but I'm not 100% sure on that.
Seeds specifically are also interesting as it's often "wasteful" to run seeds after the first time they've been materialized (if they don't change). If your seeds are pretty small then it's probably not worth worrying about but I can imagine situations where that extra time starts adding up. We are working on automatically "versioning" the dbt asset definitions, so we could in theory detect that a dbt seed hasn't changed and then make it easier to skip running seeds which have not changed since their last materialization, but this would be a bit further out.Filipe Almeida
02/14/2023, 9:47 AMNicolas Parot Alvarez
04/07/2023, 12:25 PM