Mikael Frosthage
03/03/2024, 6:02 PMScout
03/03/2024, 6:02 PMdeps
parameter in the @asset
decorator to specify the dependency. This will ensure that the downstream asset will only run after the upstream asset has completed, without needing to pass any outputs from the upstream asset to the downstream asset. Here's an example from the documentation that illustrates how to define such a dependency: python @asset def upstream_asset(): # ... logic for upstream asset ... @asset(deps=[upstream_asset]) def downstream_asset(): # ... logic for downstream asset ...
In this example, downstream_asset
depends on upstream_asset
. The deps
parameter is used to declare that downstream_asset
should run after upstream_asset
has completed. The downstream asset does not take any inputs from the upstream asset, so it's clear that it only cares about the completion of the upstream asset, not its result. This approach is suitable when you have scenarios such as executing a database query that creates or updates a table, where the downstream asset does not need to load the result of the upstream asset into memory to perform its operation. It's also useful when you want to have the reading and writing code explicitly defined in each asset, or when your assets manage I/O on their own using other libraries or tools that write to storage. Remember that defining dependencies using deps
ensures that the execution order respects the dependency graph you've defined, so the downstream asset will not execute until the upstream asset has successfully completed.