https://dagster.io/ logo
Title
a

Anton Peniaziev

03/11/2023, 7:34 PM
Hi folks, is it possible to setup airbyte assets as an upstream of another asset?
f

Fraser Marlow

03/11/2023, 9:28 PM
@ben do you have a repo for the demo of
dagster-airbyte
as you showed in the ‘Data ingestion as code’ video?

https://www.youtube.com/watch?v=JD0SNuihY-w

@Anton Peniaziev is looking for the whole source for the example to figure out how to build the dependency of dbt model on a airbyte connection.
@Anton Peniaziev the repo above is what Ben prepared for the Airbyte demo shown here -

https://www.youtube.com/watch?v=rrz2k50yMPE

- at a quick glance it demos setting up Dagster + Airbyte + bdt , so it should match your use case.
a

Anton Peniaziev

03/12/2023, 7:14 AM
Thanks a lot! @Fraser Marlow any chance you know is there an option to also define the input asset for an airbyte? I’m defining airbyte asset using
ab_assets_movies = with_resources(
    build_airbyte_assets(
        connection_id="***",
        destination_tables=["destination_tables"],
        asset_key_prefix=["asset_key_prefix"]
    ),
    {"airbyte": my_airbyte_resource},
)
or with
load_assets_from_airbyte_instance
but I cant find a way to define an AssetIn or “source_asset_key” for it. For example say I have another python asset which I want to set up as an upstream for this airbyte.
f

Fraser Marlow

03/12/2023, 5:20 PM
Sorry, I am not very technically proficient and when I try to help out, I usually cause more confusion. I will let @ben advise, or I recommend putting the question back in the main #dagster-support channel top-level thread.
b

ben

03/14/2023, 3:21 PM
Hi Anton, sorry for the delayed response! You should be able to pass
upstream_assets
to
build_airbyte_assets
, this takes a list of asset keys where you can add your source asset keys
🙏 1
a

Alejandro Henao Ruiz

06/01/2023, 5:13 AM
please @beni need an example of this, i try to do it in dagster but no success, just error after error.