19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Feel like a noob here, but what are the difference...
# ask-communityr
Roel Hogervorst
05/10/2023, 7:13 AMFeel like a noob here, but what are the differences between python_logs and compute_logs in dagster.yaml? (dagster 1.2.1, dagster-k8s 0.18.1)
✅ 1
dagster bot responded by community 1
🤖 1
a
Anthony Carapetis
05/10/2023, 7:32 AMPython logs are messages sent using python's build-in logging module (https://docs.python.org/3/howto/logging.html), Compute logs are just anything printed to stdout/stderr during op execution
❤️ 1
Anthony Carapetis
05/10/2023, 7:33 AMand those keys in dagster.yaml let you configure the capturing/storage of those two types of logs
r
Roel Hogervorst
05/10/2023, 7:36 AMThank you Anthony! It seems these two capture the same information right?
a
Anthony Carapetis
05/10/2023, 7:41 AMUsually no - most python loggers are not usually writing to the terminal, and a function/program you call from your op could be just `print`ing stuff you want to capture. Unless I misunderstand what you mean?
r
Roel Hogervorst
05/10/2023, 7:46 AMNo I think you are right, though there is some overlap. When I look at the structured logs of a run, some of the logs also go to stderr. But not all
Roel Hogervorst
05/10/2023, 7:47 AMSuddenly it becomes very clear!
Roel Hogervorst
05/10/2023, 7:47 AMthank you!
Roel Hogervorst
05/10/2023, 7:52 AMFor some reason mentally I connected the structured event logs in dagit with the button next to it: raw compute logs (with stdout, stderr). But these things are different
a
Anthony Carapetis
05/10/2023, 8:06 AMno worries! there's definitely a lot of concepts to untangle at first - still trying to get my own head around everything. And yeah, many messages will appear in multiple logs 🙂 I think the structured event logs are actually a third thing, not the same as python logs OR compute logs: https://docs.dagster.io/concepts/ops-jobs-graphs/op-events
3 Views