How can I send the logs from the containers through to a log store like Kibana?
04/04/2022, 4:50 PM
Dagster doesn’t currently have 1st class support for ELK stack, but it’s something on our radar
How do you have Dagster deployed? If it’s something like K8s, you would just be able to use the native ELK integration and all the Dagster containers would have their logs uploaded by the filebeat agents
04/04/2022, 6:05 PM
Cheers @johann was wondering if something could be done at the K8s level, that is how it's deployed
04/04/2022, 9:19 PM
Yeah we’re thinking about ways in which we can integrate better, for example showing a link in Dagit to the relevant page in Kibana
04/06/2022, 8:20 AM
Hey @johann Kibana's not currently on the same K8 cluster as Dagster, we run it on a standalone instance, does that change how I can shift the logs accross?
Tbh it'd be great if there was an option to just have them available in dagit, at least for errors. Definitely noticed that leaps and bounds of progress have been made in terms of displaying relevant errors there from an infra perspective.
I actually can see this via the UI, so thanks a lot for that!