https://dagster.io/ logo
#integration-airbyte
Title
# integration-airbyte
d

Dayten Sheffar

02/14/2023, 10:03 PM
Hi! New to Dagster and liking it so far. Having an issue I’m stumbling over for some reason. I have an asset file that connects to airbyte and pulls several named connections
Copy code
airbyte_instance = airbyte_resource.configured(
    {
        "host": {"env": "AIRBYTE_HOST"},
        "port": {"env": "AIRBYTE_PORT"},
        "username": {"env": "AIRBYTE_USER"},
        "password": {"env": "AIRBYTE_PASSWORD"},
    }
)

one_of_many_airbyte_connections = with_resources(
    build_airbyte_assets(
        connection_id="just_one_of_my_many_connection_ids",
        destination_tables=["some_table"],
        asset_key_prefix=["custom_asset_key_prefix"]
    ),
    {"airbyte": airbyte_instance}
)
and in my
module.py
I have an
__init.py__
that is choking on
assets=[...]
Copy code
defs = Definitions(
    assets=[one_of_many_airbyte_connections, ...],
    ...
)
I’m getting the following error:
Copy code
dagster._check.CheckError: Member of iterable mismatches type. Expected (<class 'dagster._core.definitions.assets.AssetsDefinition'>, <class 'dagster._core.definitions.source_asset.SourceAsset'>, <class 'dagster._core.definitions.cacheable_assets.CacheableAssetsDefinition'>). Got [<dagster._core.definitions.assets.AssetsDefinition object at 0x7f40c1099400>] of type <class 'list'>.
Frankly, not sure how to proceed. All the assets were materializing in a previous structure but we needed a more flexible workspace:
Copy code
load_from:
  # - python_file: mymodule/assets/airbyte_assets.py #OLD
  - python_package: mymodule #NEW
Could use some help, not finding what I need in the docs. Thanks!
o

owen

02/14/2023, 10:06 PM
hi @Dayten Sheffar! I think this should be a quick fix --
with_resources
(and
build_airbyte_assets
) return a
List[AssetsDefinition]
so
one_of_many_airbyte_connections
is a
List[AssetsDefinition]
. So making it a member of the list that you're passing into your defs is nesting it one layer too deep
🌈 1
quick fix would be
Copy code
defs = Definitions(
    assets=[*one_of_many_airbyte_connections, ...],
    ...
)
🙌 1
(to splat out the nested list)
d

Dayten Sheffar

02/14/2023, 10:08 PM
Guess it isn’t helping me that I have hardly used Python in two years, thank you so much @owen!
o

owen

02/14/2023, 10:08 PM
no problem blob salute
5 Views