A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis

Logs are widely used to record runtime information of software systems, such\nas the timestamp and the importance of an event, the unique ID of the source of\nthe log, and a part of the state of a task's execution. The rich information of\nlogs enables system developers (and operators) to monitor the runtime behaviors\nof their systems and further track down system problems and perform analysis on\nlog data in production settings. However, the prior research on utilizing logs\nis scattered and that limits the ability of new researchers in this field to\nquickly get to the speed and hampers currently active researchers to advance\nthis field further. Therefore, this paper surveys and provides a systematic\nliterature review and mapping of the contemporary logging practices and log\nstatements' mining and monitoring techniques and their applications such as in\nsystem failure detection and diagnosis. We study a large number of conference\nand journal papers that appeared on top-level peer-reviewed venues.\nAdditionally, we draw high-level trends of ongoing research and categorize\npublications into subdivisions. In the end, and based on our holistic\nobservations during this survey, we provide a set of challenges and\nopportunities that will lead the researchers in academia and industry in moving\nthe field forward.\n