Creating effective log formats
Most SIM / SIEM solutions require all non-textual log data (e.g. Windows Event Logs) to be rewritten into a simple textual format such as syslog. By contrast, Assuria Log Manager (ALM) is a forensically-sound log management system, because it collects logs in their original format. This allows ALM to collect arbitrary log data, rather than just logs that can readily be converted to text, and means that Assuria has developed parsers for a large number of diverse log formats, giving us perhaps a deeper understanding of the requirements of effective logs than most.
Log formats vary widely, with some variations less useful (and usable) than others. This paper (link here to the white paper) describes these variations and suggests preferred approaches for log format and content in new applications, then discusses some example log formats. The intended audience is developers of software that writes to logs and developers of log formats.