https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Slackbot

02/21/2024, 7:34 PM
This message was deleted.
s

Scout

02/21/2024, 7:34 PM
To link your Fivetran assets to your dbt assets, you need to ensure that the asset keys match between the two. Since you have defined some of your asset keys directly in the dbt source file and also have a
CustomDagsterDbtTranslator
, you can use these mechanisms to align the asset keys. Here's how you can approach this: 1. Define Asset Keys in dbt: You can customize the asset keys generated by Dagster for dbt assets by defining a
meta
key on your dbt node's
.yml
file. For example, if you want to override the asset key for a source and table as
snowflake/jaffle_shop/orders
, you would define it like this in your dbt source file:
Copy code
yaml     sources:       - name: jaffle_shop         tables:           - name: orders             meta:               dagster:                 asset_key: ["snowflake", "jaffle_shop", "orders"]
2. Custom DagsterDbtTranslator: If you want to override the asset key generation for all dbt nodes in your dbt project, you can create a custom
DagsterDbtTranslator
and implement the
get_asset_key
method. This method can be overridden to provide a custom asset key for a dbt resource. For example, to add a prefix to the default generated asset key, you would do something like this:
Copy code
python     from typing import Any, Mapping     from dagster import AssetKey     from dagster_dbt import DagsterDbtTranslator          class CustomDagsterDbtTranslator(DagsterDbtTranslator):         @classmethod         def get_asset_key(cls, dbt_resource_props: Mapping[str, Any]) -> AssetKey:             # Add custom logic to generate the asset key             # For example, adding a prefix:             return super().get_asset_key(dbt_resource_props).with_prefix("prefix")
3. Link Fivetran Assets: Ensure that the Fivetran assets have asset keys that match the ones you've defined or generated for your dbt assets. If the Fivetran connector is storing data in a table and the corresponding asset key is
public/survey_responses
, you would need to make sure that the dbt asset key matches this format. 4. Use the Custom Translator: When loading assets from your dbt project, make sure to use your custom translator so that the asset keys are generated according to your custom logic. By aligning the asset keys between Fivetran and dbt, you will be able to link the assets in Dagster, allowing you to visualize and manage dependencies between them effectively. Remember to adjust the prefixes and asset key definitions so that they match across both Fivetran and dbt.