https://dagster.io/ logo
#ask-community
Title
# ask-community
b

Ben Gatewood

05/19/2022, 4:39 AM
So....conditional asset materialisations (only materialise if the new asset would be "different" to the old one). Is there a way I'm missing?
🤖 1
if you mean dynamic partitions discussion is ongoing https://github.com/dagster-io/dagster/discussions/6913
also see my points yesterday for some problems I face at the current stage. (without full support of dagster for dynamic partitions)
b

Ben Gatewood

05/19/2022, 11:04 PM
Ah! Nice one tyvm! Will have a fiddle with this today and see how I get on
So the
multi_asset
"hack" works nicely except the condition I wish to evaluate is whether this new materialisation would be different to the current state of the asset and I...don't think I can access that here?
I mean, in this case it's a file-based asset so I can see it in the relevant place the io-manager put it but it feels a bit dirty to just read it from there?
g

geoHeil

05/20/2022, 3:42 AM
no I think the IO manager is the wrong place. The logic if it is new should go into the definition of the asset
b

Ben Gatewood

05/20/2022, 3:50 AM
Yeah the logic is in the asset but the logic I need relies on the previously materialised version of that asset which I don't think I can (or maybe just don't know how) access from within the asset definition
g

geoHeil

05/20/2022, 9:08 AM
no you do can access that by retrieving the ASSET_MATERIALIZATION Event for your specific asset key.
I.e. you could always retrieve the latest one easily
It gets more complicated when a specific one should be providable in edge cases i..e when backfilling data
b

Ben Gatewood

05/22/2022, 9:51 AM
hm ok - any guide/docs on this around? No backfilling edge cases here
6 Views