Hi All, Appreciate your help on this !

from dagster import op, job, repository, In,Out, OpExecutionContext, logger, Field
import logging
from io import StringIO

STREAM = None



@logger({
        "log_level": Field(str, is_required=False, default_value="INFO"),
        "name": Field(str, is_required=False, default_value="custom dagster logger"),
    },description="A custom Logger",)
def prepare_custom_log():
    name="custom_dagster_log"
    stream = StringIO()
    klass = logging.getLoggerClass()
    log = klass(name, level=<http://logging.INFO|logging.INFO>)
    handler = logging.StreamHandler(stream)
    log.setLevel(<http://logging.INFO|logging.INFO>)
    for handler in log.handlers:
        log.removeHandler(handler)
    log.addHandler(handler)
    return log


@op(
    name="HELLO_LOGS",
    out={"result":Out(dagster_type=str)}
)
def hello_logs(context):
    <http://context.log.info|context.log.info>("HELLO LOGS")
    return "Nothing"


@job(logger_defs={"my_json_logger": prepare_custom_log})
def demo_job():
    hello_logs()

if __name__=="__main__":
    demo_job.execute_in_process()

I have above code but I don't see dagster log name getting changed2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - RUN_START - Started execution of run for "demo_job". 2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - ENGINE_EVENT - Executing steps in process (pid: 12896) 2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - HELLO_LOGS - ENGINE_EVENT - Starting initialization of resources [io_manager]. 2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - HELLO_LOGS - ENGINE_EVENT - Finished initialization of resources [io_manager]. 2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - HELLO_LOGS - LOGS_CAPTURED - Started capturing logs for step: HELLO_LOGS. 2022-09-14 123026 -0500 - dagster - DEBUG - demo_job - ee70f313-33be-40b8-b794-ece627a2ab9a - 12896 - HELLO_LOGS - STEP_START - Started execution of step "HELLO_LOGS".

It may be a bit better to thread these for readability (EG: Post some of the output in a comment on the post). Anyways, could you specify a bit more what you wanted the behavior to be? Do you mean you want

HELLO_LOGS

to appear as

my_json_logger

?

@Zach P Thanks for the reply. my question here is what can I do to collect all logs from the custom logger?

What do you mean by collect all logs? I think this is more of a python logging than dagster question. If you wanted all your logs in say, a text file, you’d need to use a different handler. If you haven’t already check out the logging-cookbook