Has anyone tried out Scalene for profiling in Dagster? I was thinking of using it for op metrics like CPU and memory. It should work for digging into handoffs between duckdb, polars, arrow, etc - they mention C a lot but it looks like their approach works fine for rust libraries too. Maybe keeping it on for some always-on metrics if it's really lightweight enough, maybe emit some to prometheus... otherwise for profiling during dev. https://github.com/plasma-umass/scalene

This would make an interesting resource. One of the downsides of it AFAIK is it required minor code changes, but it’s been a while since I looked!