The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi! I’m currently syncing a bunch of tables with Airbyte towards my datalake, which I then process with dbt.
All of this is orchestrated with dagster, and I’m having a hard time getting them to share assets in the asset graph.

Do you mind elaborating more? What issue did you hit with “sharing assets in the asset graph”?

Screenshot 2022-07-14 at 12.17.22.png

Ah, yeah, reading that again it’s not clear at all, sorry.
The goal for me is to have the assets from airbyte be mapped to the inputs from dbt, like this:

But I’m having a hard time getting them to map to the same key - I’ve hacked something together using the dbt `node_info_to_asset_key` where I mapped it to the name that airbyte gives it, but this will break when the data doesn’t come from airbyte (and it created a hard-coupling between the dbt and airbyte jobs)

So I was wondering if I can map the airbyte assets to the dbt keys instead (since the dbt keys are “standard” inside the analytics platform), or rename asset keys that are generated somewhere