AJ Floersch02/21/2023, 1:45 PM
My reason for something like this is I would like to have a series of explicitly defined assets that can take full advantage of dagit and adhoc materializations, while also being able to dynamically execute the assets through a single job. Something that doesn't seem possible with just
@asset def render_asset_1(context): # ... do something... return something assets = [render_asset_1, ...] @op def determine_assets_to_launch(context): for asset in assets: # ... some logic on which assets are currently needed... if criteria: asset() # Run the asset @job def dynamic_asset_job(): # the job that will ultimately run on a schedule. determine_assets_to_launch()
based on an
group which will always execute every Asset, nor can it be done through something like just an op with
, which doesn't render the Asset in dagit for adhoc launches.
Saul Burgos02/21/2023, 2:54 PM
AJ Floersch02/21/2023, 2:59 PM
Saul Burgos02/21/2023, 2:59 PM
AJ Floersch02/21/2023, 3:03 PM
, but how would I do something like this outside of that?
Saul Burgos02/21/2023, 3:56 PM
claire02/22/2023, 6:48 PM
with DagsterInstance.get() as instance: ...