CERN Accelerating science

Log Analysis and Classification of CERN Control Systems

Date published: 
Monday, 1 September, 2014
Document type: 
Summer student report
Author(s): 
Z. D. Kassabov
Project Specification: CERN is running several large WinCC OA distributed sys- tems, each of them counting ~100-150 computers and several thousands of processes. The log files currently produced by these processes are only looked at very seldom to debug prob- lems. The project will consist of introducing a modern log analysis tool to analyze these log files in real time and inform the operators/experts of possible problems. The tools will also be used to extract statistics and produce reports about the control systems. The selected tool will be Elastic Search with Kibana. Abstract: This project has consisted in examining different alternatives for monitoring and ana- lyzing the log messages produced by the control system software using at CERN. Its main accomplishments have been a performance measurement of the Logstash tool un- der both realistic and extreme load scenarios, the development of a flexible and powerful performance testing tool, and the proposal of a system that automatically determines the importance that a given log message might have for an operator based on a Bayesian filter.