I have the following code;

from dagster import Definitions
from dagster_dev.assets import test_asset
from dagster_dev.loggers import custom_logger

defs = Definitions(assets=[test_asset],loggers={'custom_logger':custom_logger})

from dagster import logger

@logger
def custom_logger(init_context):
    assert False

from dagster import asset, AssetExecutionContext

@asset
def test_asset(context: AssetExecutionContext) -> int:
    <http://context.log.info|context.log.info>("Some Log")
    return 42

The logger init is never called!? I expect to see an assertion failure why do I not?

Based on the documentation, when you define a custom logger using the

@logger

decorator, the decorated function should accept an

InitLoggerContext

and return an instance of

logging.Logger

. This function will become the

logger_fn

of an underlying

LoggerDefinition

. The custom logger is then supposed to be automatically invoked whenever the methods on

context.log

are called from within job compute logic. In your code, you have defined a custom logger

custom_logger

that asserts

False

when initialized. However, you're expecting to see an assertion failure when the logger is initialized, but you're not seeing it. There are a few potential reasons why the logger initialization is not being called: 1. The logger is not correctly configured in the job or pipeline that is executing the asset. The

Definitions

object you're creating with

defs = Definitions(assets=[test_asset], loggers={'custom_logger': custom_logger})

is not by itself sufficient to ensure that the logger is used during execution. You need to make sure that the job or pipeline that includes

test_asset

is also configured to use

custom_logger

. 2. The logger might not be getting instantiated due to a misconfiguration or an issue with the execution environment. For example, if the job or pipeline is not executed, or if there is an error in the configuration that prevents the logger from being instantiated, you would not see the assertion failure. 3. The asset execution might not be reaching the point where it logs information. If there is an error in the asset code before the logging statement, or if the asset is not being executed at all, the logger would not be invoked. To debug this issue, you can check the following: - Ensure that the job or pipeline that includes

test_asset

is configured to use

custom_logger

. This typically involves setting the

logger_defs

argument in the job definition to include your custom logger. - Verify that the job or pipeline is being executed correctly and that there are no errors preventing the execution from reaching the point where

<http://context.log.info|context.log.info>

is called. - Check the execution environment and configuration to ensure that there are no issues preventing the logger from being instantiated. If you've confirmed that the job or pipeline is set up correctly and the asset is being executed without errors, but you're still not seeing the assertion failure, it may be necessary to look into the specific details of your execution environment or consider other potential issues that could be affecting the logger initialization.

@Scout In response to; "to use

custom_logger

. This typically involves setting the

logger_defs

argument in the job definition to include your custom logger." I do not make use of an explicit Job - I am on;y using the @asset decorator - so how do I set the

logger_defs

In Dagster, when you're using assets without an explicit job definition, you can still configure loggers by using a repository definition that includes your assets and loggers. Here's how you can do it: 1. Define your custom logger using the

@logger

decorator, as you have done. 2. Create a repository that includes your assets and logger definitions. 3. Use the repository in Dagit or when executing your assets programmatically. Here's an example of how you might define a repository with your custom logger and asset:

python from dagster import repository, asset, logger, AssetExecutionContext  # Define your custom logger @logger def custom_logger(init_context):     assert False  # Define your asset @asset def test_asset(context: AssetExecutionContext) -> int:     <http://context.log.info|context.log.info>("Some Log")     return 42  # Define a repository that includes your asset and custom logger @repository def my_repository():     return {         "assets": [test_asset],         "loggers": {"custom_logger": custom_logger},     }

When you load this repository in Dagit, it should automatically pick up the custom logger definition. When you execute your asset, the custom logger should be initialized, and you should see the assertion failure as expected. If you're executing your assets programmatically, you can use the repository definition to configure the execution context with the custom logger. Here's an example of how you might do that:

python from dagster import execute_pipeline, build_pipeline_definition  # Assuming you have a repository as defined above pipeline_def = build_pipeline_definition(my_repository)  # Execute the pipeline with the custom logger result = execute_pipeline(pipeline_def)

Make sure that your repository is correctly configured and recognized by Dagster, and that you're using the correct execution commands that take the repository's configuration into account.

@Scout but I am not using repository derictly either - I have specified the loggers in the Definitions like;

defs = Definitions(assets=[test_asset],loggers={'custom_logger':custom_logger})

If you are using the

Definitions

class to configure your assets and loggers, and you are not seeing the logger initialization being called, it could be due to the way you are executing your assets. The

Definitions

class is used to implicitly create a repository under the hood, which is then used by Dagster's tooling. To ensure that your custom logger is being used when executing your assets, you should execute your assets in a context that respects the

Definitions

configuration. This typically involves using Dagster's command-line interface (CLI) or Dagit UI, which will automatically pick up the

defs

variable from your module and use it to construct the execution environment. Here's what you need to do: 1. Make sure your module (let's call it

my_module.py

) with the

Definitions

is structured like this:

python
from dagster import asset, logger, Definitions, AssetExecutionContext

# Define your custom logger
@logger
def custom_logger(init_context):
    assert False

# Define your asset
@asset
def test_asset(context: AssetExecutionContext) -> int:
    <http://context.log.info|context.log.info>("Some Log")
    return 42

# Create Definitions with your asset and custom logger
defs = Definitions(assets=[test_asset], loggers={'custom_logger': custom_logger})

2. Use the Dagster CLI to execute your assets. If you're using the CLI, you can run the following command:

bash
dagster asset execute -f my_module.py

This command tells Dagster to execute the assets defined in

my_module.py

, and Dagster will use the

defs

variable to construct the execution environment, including the custom logger. 3. If you're using Dagit, simply start Dagit while pointing to the directory containing

my_module.py

, and Dagit will automatically load the

defs

variable and use it to construct the execution environment.

bash
dagit -f my_module.py

When you execute your assets through these methods, Dagster should use the custom logger defined in your

Definitions

, and you should see the assertion failure as expected. If you are still not seeing the logger initialization being called, you may want to verify that there are no issues with the way you are executing your assets or with the environment in which Dagster is running. Additionally, ensure that there are no errors in your asset or logger definitions that could be preventing the logger from being initialized. Loggers | Dagster button Dagster log handler re-logs messages to ... button Dagster Docs button "DagsterInvariantViolationError: Provide... button Thank you for your feedback!