Hello, I know if I want to output say 5 rows from ...
# announcements
d
Hello, I know if I want to output say 5 rows from a pandas dataframe in dagit's log output, I could use the EventMetaData feature of dagster (per https://github.com/dagster-io/dagster/issues/3234), but I would think I should be able to just output it using the context.log.info() function. I get an error if I do:
<http://context.log.info|context.log.info>(df_final.sample(n=5).transpose())
Error message:
dagster.core.errors.DagsterExecutionStepExecutionError: Error occurred while executing solid "feature_selection":
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\core\execution\plan\execute_plan.py", line 190, in _dagster_event_sequence_for_step
for step_event in check.generator(step_events):
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\core\execution\plan\execute_step.py", line 311, in core_dagster_event_sequence_for_step
for user_event in check.generator(
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\core\execution\plan\execute_step.py", line 70, in _step_output_error_checked_user_event_sequence
for user_event in user_event_sequence:
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\core\execution\plan\execute_step.py", line 596, in _user_event_sequence_for_step_compute_fn
for event in iterate_with_context(
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\utils\__init__.py", line 363, in iterate_with_context
return
File "D:\Python38\lib\contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "D:\Python38\envs\ashli_dev\lib\site-packages\dagster\core\errors.py", line 197, in user_code_error_boundary
raise error_cls(
The above exception was caused by the following exception:
dagster.check.ParameterCheckError: Param "msg" is not a str. Got 681 214 212 230 521
Parch 0 0 0 0 0
Fare 76.7292 7.75 7.25 83.475 7.8958
Embarked C Q S S S
Sex male male male female male
Name Hassab, Mr. Hammad Kiernan, Mr. Philip Perkin, Mr. John Henry Harris, Mrs. Henry Birkhardt (Irene Wallach) Vovk, Mr. Janko
Age 27 NaN 22 35 22
Survived 1 0 0 1 0 which is type <class 'pandas.core.frame.DataFrame'>.
However, I get no error if I do:
<http://context.log.info|context.log.info>(f"5 random rows: \n {df_final.sample(n=5).transpose()}")
I get the output that I expect. With Python's standard lib logger, I can output a partial dataframe without using f-string or having to convert to string.
So I guess what I'm saying, I expected that I could just pass in a "bare" dataframe to the context.log.info() as I was using my experience with Python's standardlib logger as reference. But I guess I can not.
m
the loggers don't coerce with
str
, but maybe they should
would you mind opening an issue?
d
Hey @max, You're right, loggers shouldn't have to coerce to str, Im with dagster's philosophy with being type strict, so I am ok with this behavior as is.