When I issue context.log_info("some string"), I ca...
# ask-community
w
When I issue context.log_info("some string"), I can see in on Dagit, but where is it stored in the database (I use PostgreSQL backend)?
a
in the
event_logs
table
w
That's the first place I look, but I can't seem to find it. What event type would it be in?
a
null
, itll be in the
event
col json payload
w
The actual value that I'm searching for is stored in a file whose path is in that "event" field??? Why?
a
I don’t follow.
context.log
should result in a row in the
event_logs
db with a json serialized
EventLogEntry
object in the
event
column containing the contents of the logged string in the
user_message
field.
w
@alex this is the content of the "user_message" field in my case:
Copy code
"user_message": "Writing file at: /home/de1/dagster_f45/storage/b7066d95-83a8-496e-af7f-5d312cb17838/insights_refresh_time/last_insights_refresh"
And sure enough, if I follow that path and see the content of that file, the actual message I am looking for is in there.
Did I miss a setting that would put that message in the db instead of a file like that?
a
what does your
@op
look like? It appears that this is the
context.log.debug
from the
fs_io_manager
resource which would correspond to returning a value from your op. If you explicitly logged the value yourself in the op i would expect it to to be in the db directly
w
The line that produced that event is this:
Copy code
<http://context.log.info|context.log.info>(some_str_var)
a
do you then return
some_str_var
? I think you are finding the log entry output by https://github.com/dagster-io/dagster/blame/master/python_modules/dagster/dagster/core/storage/fs_io_manager.py#L117
w
Yes, I do return that same variable. So where can I find the event associated with the context.log.info, not the return?
I found it!
It's the previous status=null entry in the table.
So the question now is, how do I distinguish it from the row produced by the return value?
I notice that the event produced by the return value has level:10, and the one by the context.log.info has level:20. What does level indicate here? @alex
a
log.info vs log.debug
w
Thank you, so now I can filter with level=20 to find all my log.info statements
a
what exactly is the overall goal here? would it be better to just explicitly write these values to your own table?
w
Why would I need to create a separate table if I can query it right within the context of the value?
My goal is to store some data about the ETL run that I may want to query one day as a metric.
I know that I can do that using AssetMaterialization, but I'm exploring if I can also query log.info because I use those too.
And thanks to you, now I know how to query them. So Thank You! 🙂
👍 1
@alex Just want to make sure you get my Thanks! I appreciate you helping me.
👍 1
a
Why would I need to create a separate table if I can query it right within the context of the value?
it’s theoretically possible some future version may change how those logs are stored