Alec Koumjian02/21/2023, 4:10 PM
. Those parameters would include things like the user account the job is for and basic inputs. I'd like to store the result of those jobs in files in cloud bucket storage (same as we do with our assets). I'm wondering if the jobs themselves should be yielding AssetMaterializations with dynamic asset keys, or if we should not use assets and handle the I/O directly. It would also be nice to use the input parameters such as requesting user account id against concurrency limits. I think this can be done by adding tags at runtime, does that sound right?
when using the
constructs or would this have to be done with an
and an explicit
claire02/22/2023, 10:49 PM
and yielding an
objects are meant to deterministically output a materialization for the given asset, so using the `@op`/`AssetMaterialization` approach is likely much more straightforward. If you wanted to dynamically specify an asset key to materialize with an
, you'd have to dynamically generate an
and provide a name for the asset upon job run.
Alec Koumjian02/23/2023, 5:13 PM
which I believe is somehow default behavior with `@op`s?
claire02/23/2023, 5:32 PM