https://dagster.io/ logo
Join Slack
Powered by
Hi, How do I connect assets from Fivetran with DBT...
# integration-dbt
b
Hi, How do I connect assets from Fivetran with DBT. Both of them do not get loaded with @asset. The examples I see in the docs were not really clear for me. I tried something like this: https://medium.com/starschema-blog/dagster-airbyte-dbt-how-software-defigned-assets-change-the-way-we-orchestrate-ac70bb29d640 But the api for fivetran seems to work differently. Can someone give me an example and explain it like I’m 5? Do I have to connect each table one by one or is there an automatic way that finds the connection between each of the tables in Bigquery (DBT) and Fivetran?
✅ 1
For my understanding the key_prefix need to match for dbt and fivetran for that asset. Is there a way I can change the key_prefix coming from fivetran. I only see an option to change the prefix I don't see an option to replace it. I want to replace google_ads_123 coming from the connector name to google_ads.
Screenshot 2023-01-25 at 01.26.13.png
Screenshot 2023-01-25 at 01.27.46.png
Found the solution:
Copy code
def connector_to_asset_key_fn(_, item_name):
    config = {
        "google_ads_123": "google_ads",
        "linkedin_123": "linkedin"
    }
    path = item_name.split(".")
    path[0] = config[path[0]]
    return AssetKey(path=path)
Anf then:
Copy code
fivetran_assets = load_assets_from_fivetran_instance(
    fivetran=fivetran_instance,
    key_prefix=["dbt_source"],
    connector_to_asset_key_fn=connector_to_asset_key_fn
)
❤️ 1
n
If you only had one schema, say 
google_ads
, another approach might be something like this:
Copy code
connectors_to_include = ["my_conn"]

fivetran_assets = load_assets_from_fivetran_instance(
    fivetran_instance,
    connector_filter=lambda meta: meta.name in connectors_to_include,
    connector_to_asset_key_fn=lambda _, table: AssetKey(path=table.split(".")[-1]),
    key_prefix=["dbt_source", "google_ads"],
)
8 Views
Open in Slack
PreviousNext
Powered by