This is a bit of a left-field question and I know ...
# ask-community
r
This is a bit of a left-field question and I know Dagster don’t own the Openlineage integration but throwing this here in any case…it’s broken on the EventRecordsFilter piece. There’s not an active maintainer. I guess, from error logs, it now needs a dagster event type as a required attribute and the current implementation of the integration doesn’t offer that up. It filters on event type later (or rather, checks for specific events). I’m trying to not have to rewrite the whole thing and was wondering if there’s a way to get the event logs without requiring event type. Here’s the broken code. https://github.com/OpenLineage/OpenLineage/blob/main/integration/dagster/openlineage/dagster/utils.py - get_event_log_records…(called and processed from here: https://github.com/OpenLineage/OpenLineage/blob/main/integration/dagster/openlineage/dagster/sensor.py) .
🤖 1
s
Hi Ray, Thanks for this report. Like you noticed, we don’t own the OpenLineage integration so can be of limited help here. It looks like there is already an issue though: https://github.com/OpenLineage/OpenLineage/issues/2043 I recommend posting on that issue. FWIW We export an enum
DagsterEventType
from dagster, looks like the fix might be as simple as providing the appropriate value from this enum, but I lack context on OpenLineage to know what that should be.
r
Yes, thank you, I’d seen that post and concluded that if anything was going to get done, I’d probably need to do it, which is fine, since there was no active maintainer (dagster or otherwise). It seems to be looking for a range of events - PIPELINE/STEP and will maybe start to look from that perspective. Thanks again!