Could the 'step skipped' logs be demoted to DEBUG level, similar to the started/succeeded messages? In console output with the level set to INFO, I'm getting lots of them. Like

dagster - INFO - Skipping step data_pipe_graph_...

. Similar, when there's an error, I get a flood of error logs for the downstream ops to the one that hit the error. Could those move to DEBUG? Like

dagster - ERROR - Dependencies for step list_test_ids_by_pipe_for_metadata failed: ['_validate_config']. Not executing.

I don’t think we have a super handy way to config this, but you can customize a logger to tune the log level per your need. Here’s an example: https://docs.dagster.io/concepts/logging/loggers#customizing-loggers

I did set the console log level following this example to

INFO

, by doing this:

loggers:
 console:
  config:
   log_level: INFO

I guess I could create a custom logger, set its level to

INFO

, and add a filter to remove these specific messages. But what I was wondering was whether the Dagster team would consider changing the level of the "Skipping step" and "Dependencies for step ... failed" messages to DEBUG.

Ah I meant that you could potentially custom a logger that looks up certain message and “downgrade” the log level manually (as a workaround).

Gocha, makes sense.

I could imagine making “Skipping step” a DEBUG could cause some confusion for some users, as DEBUG isn’t shown by default and when skipping is a surprise it could be misleading. As for the error message “Dependencies for step …“, I don’t have a strong opinion and am leaning towards DEBUG - technically it’s “not executing” which isn’t an error.

Yeah, definitely don't want to make silent decisions that surprise people. My reason to argue for DEBUG would be that there's really clear UI treatment for skipped steps, and the console message seems like a backup message stream for 'wait, what exactly is happening' similar to start/succeeded/etc DEBUG messages. If I filter to INFO, these two messages (skipped and dependencies failed) actually seem to be the only two Dagster processing messages left, so they seem more like outliers than part of a clear description of execution. But I'm definitely favoring my own use case here. (: