Jason Gofford
03/05/2024, 10:16 AMScout
03/05/2024, 10:16 AMkey_prefix
, you would typically use the @asset
decorator and specify the key_prefix
argument. When you want to define dependencies between assets, you can use the deps
argument within the @asset
decorator to specify upstream assets. Here's an example from the documentation that shows how you might define an upstream asset with a key_prefix
and a downstream asset that depends on it: python from dagster import asset, AssetKey @asset(key_prefix="prefix") def upstream(): # ... upstream computation ... @asset(deps={"upstream": AssetKey(["prefix", "upstream"])}) def downstream(upstream): # ... downstream computation that depends on upstream ...
In this example, the upstream
asset is defined with a key_prefix
of "prefix"
, which means its full asset key will be AssetKey(["prefix", "upstream"])
. The downstream
asset then specifies its dependency on the upstream
asset by providing a dictionary to the deps
argument, where the key is the argument name ("upstream"
) and the value is the AssetKey
of the upstream asset. If you are integrating with dbt and want to define a dbt asset as an upstream dependency of a non-dbt asset, you can use the DbtManifest
class to fetch the asset key of a given source or model. This allows you to reference dbt models directly without having to manually construct the asset key. Here's an example from the documentation: python from dagster_dbt import dbt_cli_resource, dbt_assets manifest = dbt_cli_resource.configured({"project_dir": "path/to/dbt/project"}).manifest @dbt_assets(manifest=manifest) def my_dbt_assets(): ... @asset(non_argument_deps={manifest.get_asset_key_for_model("my_dbt_model")}) def downstream_asset(): # ... downstream computation that depends on a dbt model ...
In this example, my_dbt_assets
is a collection of dbt assets loaded from a dbt project, and downstream_asset
is a non-dbt asset that depends on a dbt model named "my_dbt_model"
. The non_argument_deps
argument is used to specify the dependency on the dbt model by using the get_asset_key_for_model
method from the DbtManifest
object. Please note that the exact syntax and methods may vary depending on the version of Dagster and dagster-dbt you are using, so it's important to refer to the specific documentation for your version to ensure you are using the correct API.