Hey guys I would like to have 2 jobs materializin...
# ask-community
r
Hey guys I would like to have 2 jobs materializing assets in this way: 1. every 24h materialize a big asset 2. every 15m materialize a small asset depending on the big asset, but do not rematerialize it if the first job succeeded (and fail if it failed) Is there a clean way to do this?
s
from what I understand from here it should work just fine https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#partition-dependencies So every partition that is depending on the big asset (24*4 per day) should use already materialized asset and it should fail if big_asset have failed status
s
but do not rematerialize it if the first job succeeded (and fail if it failed)
are you able to clarify what you mean by this a little further? let's say the big asset is rematerialized at 1 am and succeeds at 1:10. are you saying that you wouldn't want to materialize the small asset at 1:15?
r
I would want to materialize the small asset at 1:15, but not the big one. If I exclude it (the big asset) from the asset job and the big asset materialized yesterday, but failed today, the small asset will use yesterdays materialization, right? On the other hand, if I include it in the asset job it rematerializes the big asset aswell. I suppose what I want could be achieved with something like this (I think):
Copy code
@asset
big_asset():
  return

@asset
small_asset(big_asset):
  assert latest_materialization_of_big_asset_succeded()
  return
Then only use the small_asset in the asset job. @Szymon Zaborowski you mean that I make my small_asset a partitioned asset? And I wouldnt have to use a custom IOManager to store only the latest partition on disk?