hey all, trying to figure out what the best/clean...
# announcements
hey all, trying to figure out what the best/cleanest way to do the following would be: I have my pipeline configured to automatically send intermediate materializations to s3, works great saves to
, I'm also using
to create summary statistics on dataframes (and other types) I would like to send these to s3 as well so I have all of my intermediate results and
in one place, I suppose I could do this with materializations, but then I'm already automatically materializing them, and I'm already creating the
in the type checks, and would feel hacky to re-add them to a materialization loving dagster so far, and it's crazy how fast you guys are improving it, started using it like 3 weeks ago and every day it feels a little more awesome to use
🎉 1
hmm ya this is tough
this is a little crazy - but you can get at the structured event stream from a custom logger, you could use that to grab the
off of the type checks and send it somewhere
this is something were planning on writing some better support for - for now it would look a little like this:
Copy code
import logging
from dagster import logger
class EventListener(logging.Handler):
    def emit(self, record):
        event = record.dagster_meta.get('dagster_event')
        if not event.is_dagster_event:

        dagster_event = event.dagster_event

        # bad name - just means output event
        if dagster_event.is_successful_output:
            # send dis

def logger_based_event_listener(init_context):
    klass = logging.getLoggerClass()
    logger_ = klass(name, level=level)
    return logger_
would feel hacky to re-add them to a materialization
ya its not great but it might be your best bet
thanks, gonna play around with this for now, awesome that your planning on adding this later!