filter-syslog

"I see the world didn't end yesterday."
"Are you sure?"

Alan Moore, Watchmen

Warning

This script is orphaned. Although I believe it is still useful, I no longer use it and am no longer maintaining it. If you would like to pick up maintenance of this script, please feel free. Contact me if you would like this page to redirect to its new home.

Description

I wrote this program because we had hundreds of systems all merrily saving their syslog files nightly into AFS to absolutely no purpose whatsoever apart from wasting disk space because no one looked at them. We had all the information we'd ever want to check for problems, but since we didn't have anything in place to do that, it was just sitting there, saved in case we could do something with it later.

We fixed that problem with this program by running via out of newsyslog nightly. It's a fairly simple filter, much simpler than other, more complex programs like logcheck, but that also makes it easier to teach someone how to configure. It allowed us to stop archiving regular syslog files in AFS, since we got separate summaries of the useful information. It supports parsing either syslog or Apache error log files, matching lines against a set of regular expressions, and mailing out any lines that don't match.

filter-syslog isn't really designed for individual systems; for that, I recommend instead installing the Debian logcheck package. It's more for deploying standard syslog filtering across hundreds of servers.

This is not a security tool. It does not monitor logs in real time like swatch. If someone breaks into the system and removes the incriminating evidence from syslog, this won't catch it. Rather, this is designed more to be run periodically to pick up things like disks that are starting to complain, unusual activity or software failures that otherwise might not have been noticed, and just things that were happening on the system that you didn't know about. It works best with a syslog.conf file that directs all of the logs to one file, or when run as part of the log rotation process on the just-rotated log.

For a similar (but obsolete) program to filter logs managed by multilog, see multilog-watch.

Requirements

Perl 5.6.0 or later, and a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that supports standard sendmail options and can be used to send the filtered report via mail.

Download

The program:

filter-syslog 2.3 2012-05-19 Download

A Debian package is available from my personal repository.

Documentation

License

Copyright 2002, 2003, 2004, 2006, 2007, 2009, 2010, 2011, 2012 The Board of Trustees of the Leland Stanford Junior University.

This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself. This means that you may choose between the two licenses that Perl is released under: the GNU GPL and the Artistic License. Please see your Perl distribution for the details and copies of the licenses.

Last modified and spun 2014-08-10