I'm working roughly from the MDS demo in the example folder of the git repo. I feel like I should be able to strip out the pandas_io_manager, because DBT should just be executing SQL statements against the database, but if I try to remove it I get:

docker-dev-dagster-pipelines     |   File "/usr/local/lib/python3.9/site-packages/dagster/core/asset_defs/asset_group.py", line 755, in _validate_resource_reqs_for_asset_group
docker-dev-dagster-pipelines     |     raise DagsterInvalidDefinitionError(
docker-dev-dagster-pipelines     | dagster.core.errors.DagsterInvalidDefinitionError: Output 'study_list_completion_mvw' with AssetKey 'AssetKey(['raw', 'study_list_completion_mvw'])' requires io manager 'pandas_io_manager' but was not provided on asset group. Provided resources: ['dbt', 'io_manager']

Could anyone explain to me why the SQL side of the DBT integration requires an IO manager?

hi @George Pearse! good question -- this io manager doesn't actually do anything on the dbt side (which is confusing). It's purely used to tell downstream assets that take these dbt models as input how to load them. If you don't have anything downstream of the dbt models, then there's no need for an io manager.

Fair enough, sure I will use its core functionality soon. My only concern was if I was unnecessarily loading lots of data into python between the DBT models.

gotcha! 👍

it's surprising to me that you're seeing that error @George Pearse. Did you make sure to remove

io_manager_key="pandas_io_manager"

in the invocation of

load_assets_from_dbt_manifest

?

Ah think you've got it