Hey everyone! 👋 I’ve got a quick question about logging inside Dagster. Coming from Prefect, I’m used to “externalizing” some portions of code as foreign functions (understand “outside of the Prefect scope”) in order to reuse functionalities from within several tasks, which helps minimizing the overall code redundency. Basically, I’m just calling generic functions from within several tasks, like filesystem-related operations, etc. In order to log stuff from within these foreign functions, Prefect exposes the main logger as a contextual object, so you can use it outside of a Prefect-declared task like this:

# Prefect example
import logging
import prefect

@task
def call_a_foreign_function():
    log_from_a_foreign_function()

def log_from_a_foreign_function():
    logger = prefect.context.get("logger", logging.getLogger("default_logger_if_no_prefect_context_is_found"))
    <http://logger.info|logger.info>("It works! I can still log to the main logger even if I'm a foreign function!")

I tried to do something like this in order to do the equivalent within Dagster:

# Dagster equivalent
import logging
from dagster import solid

@solid
def call_a_foreign_function():
    log_from_a_foreign_function()

def log_from_a_foreign_function():
    logging.getLoggerClass()("dagster").warning("Can I speak to the UI like this?")
    logging.getLogger("dagster").warning("Can I speak to the UI like this?")

Sadly, it does not seem to work, I’m only able to log to the Dagit console (not the UI). Does Dagster have a cool way to log to the UI from “outside” of solids ?

Hi Nelson! there's a context argument to solids that exposes a logger in a fairly similar way - there are some code examples here: https://docs.dagster.io/overview/logging/logging

Thanks for the quick answer @daniel. Honestly, I find it a bit cumbersome to just pass around the

context.log

object to every foreign function, is there another way to “retrieve the logger from the other side” ?

I don't think we currently expose it as a global unfortunately 😕 I believe one reason for this is that in dagster loggers are pipeline-scoped and can be customized depending on configuration on your run, so there might be situations in which there might not be a clear answer for the one true logger to use without more information about the context in which its being called.

It does make sense, thanks for the answers 🙂

You could definitely create and register a logging.Handler at the top of your solid if you wanted that

this is an open question tbh -- very interested to hear from you that it seems easier to

import logging

and log that way

Sorry, I’m late to the party! To be honest, the custom context loggers do seem straightforward to use. There may be some limitations (like the case I brought above) but I don’t see any problem with the current way of doing things if you stick with the “Dagster logic”. In the long run, and as users experiment with the project, I think it will be beneficial to opt for a more open implementation.