Log

AssetMaterialization

from a failure hook I can see that it's possible from an op by calling

OpExecutionContext.log_event

, but

HookContext

doesn't provide the same mechanism. Is it reasonable to expect to be able to log an

AssetMaterialization

from a failure hook, or is there some good reason why this is non-sensical?

Hey @Stefan Adelbert! I think it would be helpful to hear more about your use case and why you need to materialize an asset from a failure hook. In general, the asset materializations should happen in an op, since that is the unit of computation in dagster. In the typical case, you'd try to materialize your asset in the op, and if that materialization failed, then you'd go into the failure hook. in that case it doesn't usually make much sense to try and materialize the asset again since it would likely fail for the same reason as in the op. the failure hook is more to allow you to send an alert or do any clean up operations that may be necessary if your op fails. Definitely let me know what you are trying to achieve with your set up and we can go from there!

@jamie Thanks very much for your reply. I'm using

dagster

to run data processing and automation tasks. The automation tasks tend to involve interacting with a web browser - I'm using selenium to interact with a headless firefox instance available to ops as a resource. I have the ability to take screenshots of the headless browser, which is particularly useful for fault finding issues, especially given that the browser is headless. I've been materializing each screenshot taken for a particular run against the same asset, i.e. there is an browser screenshot asset for each run with potentially several materializations, one for each screenshot. If there is some error in an op, I'd like to materialize a screenshot showing the state of the browser at the time of the failure, hence my query. I get that this is probably not the intended use of asset materializations. I'm keen to hear your thoughts.

sorry for the late reply on this. you use case makes sense as to why you'd want to materialize on op failure. One thing you might try instead is catching the exception that causes op failure in a

try except

block, and materializing the asset there, then reraise the error so the op still fails

Thanks for the reply, @jamie. I understand your solution, but really I was hoping for something more general that could be applied to some ops and not to others, without having to modify the ops themselves.

ok I see your point. definitely feel free to open a github issue requesting this feature and we'll get it into our backlog