Shane Keller
03/06/2024, 10:05 PMScout
03/06/2024, 10:05 PM@asset
decorator in Dagster is used to define software-defined assets. When defining assets, you may need to specify dependencies to indicate how assets relate to each other. In Dagster, there are two ways to specify dependencies for assets: deps
and ins
. 1. `deps`: The deps
parameter is used to specify dependencies between assets. When you use deps
, you directly pass other asset functions to indicate that the asset being defined depends on the outputs of those assets. This is a newer API that allows for better type checking and error detection at development time because it directly references asset functions. 2. `ins`: The ins
parameter is an older way to specify dependencies and has been deprecated in favor of deps
. It used to rely on string matching asset names, which could lead to typos and errors that were only caught at runtime. The ins
parameter allowed you to specify dependencies by providing a dictionary where the keys are the input names and the values are instances of AssetKey
or AssetIn
. The documentation suggests using deps
over ins
because it provides a more robust and error-resistant way to define dependencies between assets. The deps
parameter accepts assets directly, which allows in-editor type checkers to detect errors, whereas ins
relied on string matching and was more prone to runtime errors. Here's an example from the documentation showing the transition from the old API to the improved API: Old API using non_argument_deps
(similar to ins
): python @asset(non_argument_deps={"iris_dataset"}) def iris_setosa(duckdb: DuckDBResource) -> None: ...
Improved API using `deps`: python @asset(deps=[iris_dataset]) def iris_setosa(duckdb: DuckDBResource) -> None: ...
In this example, iris_dataset
is another asset that iris_setosa
depends on. The deps
parameter replaces the use of non_argument_deps
, and non_argument_deps
has been marked deprecated. The deps
parameter allows for better error detection and code clarity.