Selim Raboudi
02/23/2024, 2:59 PMScout
02/23/2024, 3:00 PMbuild_airbyte_assets
function to create assets for the tables created by an Airbyte sync operation. If you want to make another asset depend on only one of the output tables from an Airbyte connection, you can specify the dependencies using the deps
parameter when defining your asset. Here's an example from the documentation on how you might define an asset that depends on specific Airbyte output tables: python from dagster_airbyte import build_airbyte_assets from dagster import AssetGroup, asset airbyte_assets = build_airbyte_assets( connection_id="some_connection_id", destination_tables=["users", "orders"] ) @asset def other_asset(users): # ... update the other_asset table based on the 'users' table my_group = AssetGroup( assets=airbyte_assets + [other_asset], resource_defs={"airbyte": my_airbyte_resource} )
In this example, other_asset
is an asset that depends on the users
table, which is one of the output tables from an Airbyte connection. The users
argument in the other_asset
function represents the dependency on the users
table asset. When you define your asset, you can specify the dependencies by passing the required asset keys to the deps
parameter. If you only want to depend on a single table, you would only include that specific table's asset key in the deps
list. Keep in mind that the asset keys must match the keys of the Airbyte assets for the dependencies to be correctly established. If you have set a custom asset_key_prefix
or used a custom function to define asset keys for your Airbyte assets, ensure that the keys you use in the deps
parameter match those custom keys.