Is there a recommended approach to stream all types of pipeline event types per run I.e Run Start / Step Start?

hi @Matt O -- by "stream", do you mean stream them to some external system? You can query the event database for different event types, using

context.instance.get_event_records

(inside either an op or a sensor).

Hi @owen! Yes, I want to publish events to a queue as they happen

Got it -- there's no direct way to stream these records, but you could definitely run that query on a pretty tight loop. One option would be to create a single-op job which runs that query and publishes all new records to your queue, and run that job every 30 seconds (or faster, if you have more real-time requirements).

Got it, I'll look into this and test it out. Thanks for the insight 👍👍