https://dagster.io/ logo
#ask-community
Title
# ask-community
f

Flavien

08/10/2022, 6:56 AM
Hey team 👋 I'm using the
dagster-dbt
integration and specifically
load_assets_from_dbt_manifest
. This function defines two parameters: • `select`: a dbt selection string for the models in a project that you want to include. • `selected_unique_ids`: the set of dbt unique_ids that you want to load as assets. They appear quite similar so I guessed that
select
limits what gets executed by
dbt run
while
selected_unique_ids
selects what gets exposed as an asset in Dagster. Is it correct? I wanted to test that assumption so I created this simple DAG in dbt
a --> b
and loaded the asset this way:
Copy code
assets = load_assets_from_dbt_manifest(manifest, select="+b", selected_unique_ids={"model.dbt_project.b"})
but I got the following error:
Copy code
dagster._core.errors.DagsterInvariantViolationError: Core compute for op "run_dbt_dbt_project_aeba1" returned an output "a" that does not exist. The available outputs are ['b']
After digging into the integration code, I saw that the op does not account for the
selected_unique_ids
and yield materialization for every dbt node. Bug? An other related question is: why is the
selected_unique_ids
parameter not present in the definition of
load_assets_from_dbt_project
?
o

owen

08/10/2022, 4:55 PM
hi @Flavien! the
selected_unique_ids
parameter is generally is not useful to set yourself (and the docs should be updated to reflect this). basically, it represents the "resolved" unique ids from a dbt select statement, and it should either be
None
, or exactly match the set of unique ids specified by the
select
parameter. It mostly exists for legacy reasons -- originally
load_assets_from_dbt_manifest
did not support the
select
parameter (for various reasons), so this was an alternate way for specifying that selection. Nowadays, the select parameter is generally a much more convenient way of specifying that information.
both of those parameters refer to the set of models you want to load as assets (there's no distinction between the assets you see in the UI and the models that will be run)
f

Flavien

08/12/2022, 8:36 AM
Thank you @owen for the response and also congratulations for the 1.0 release. It would be quite useful to also have an
exclude
parameter. I saw you created an issue last month to do just that. Are you open to contributions?
o

owen

08/12/2022, 4:11 PM
hi @Flavien! That's definitely on our radar, and we're absolutely open to contributions
4 Views