Alejandro A
02/09/2024, 6:22 PMScout
02/09/2024, 6:22 PMDynamicOutput
objects from an op with DynamicOut
declared as its output. However, when working with software-defined assets, the concept of dynamic outputs is not directly applicable in the same way. Software-defined assets are typically defined with a static set of outputs, and currently, there is no direct way to dynamically generate assets based on dynamic outputs from an op or graph. The documentation mentions that it's not possible to map a dynamic output to a certain output of a graph-backed asset, and there is no support for subsetting graph-backed assets dynamically. If you have a job that relies on dynamic outputs and you want to transition to using software-defined assets, you might need to reconsider the design. One approach could be to use partitions to represent the dynamic aspect of your computation. Partitions allow you to define slices of an asset that can be materialized independently, which might be a way to achieve a similar effect as dynamic outputs. Here's a general approach you might consider: 1. Define your assets with partitions that represent the dynamic aspect of your data. 2. Use the partition keys to control the computation and materialization of each slice of your asset. 3. Instead of yielding dynamic outputs, your asset computation logic would handle each partition accordingly. Keep in mind that the documentation suggests using partitions for large numbers of objects with the same structure to make operations like backfills easier and the UI more scalable. Since the documentation does not provide a direct example of migrating an op with dynamic outputs to a software-defined asset, you may need to explore alternative designs that fit within the current capabilities of software-defined assets in Dagster. If you require further assistance with a specific implementation or if you have more detailed requirements, please provide additional information so we can help guide you more effectively.