Snoopit

Simple tool for monitoring process log files for specified events and then generating basic notifications. This is an extensible and data driven solution. It provides a single location to manage log scraping duties.

Installation

Add this line to your application's Gemfile:

gem 'snoopit'

And then execute:

$ bundle

Or install it yourself as:

$ gem install snoopit

Usage

Typical use is via the command line with a JSON specification file.

Usage: snoopit [options]
    -s, --snoopers snoopies.json     File contains one or more regular expressions to locate a line of interest in a file
    -S, --snooper snooper_name       Only use the named snooper. This option can be used more than once to use several snoopers.
    -t, --template                   Generate a template snoopies.json file to stdout
    -T, --tracking                   Enable log file tracking using file ./snoopit_db.json
    -f, --tracking-file file_name    Specify a different tracking file name and location instead of the default ./snoopit_db.json
    -j, --json                       Generate output in json
    -J, --pretty-json                Generate output in pretty json
    -N, --no-newline                 Do not output new line between found items
    -n, --enable-notifications       Enable notifications
    -l, --line-numbers               show line numbers
    -v, --verbose                    prints out file name, matched line number
    -h, --help

Basic Snoopers specification file

Snooper.json

    {
      "snoopers" : {
        "AppServer1" : {
              "snoop" : "/opt/app_servers/app_server1/log/app.log",
              "sniffers" : [
                {
                  "comment" : "Failed to communicate with remote sync server",
                  "regexp" : "ERROR: Sync",
                  "lines" : {
                    "before" : 1,
                    "after" : 1
                  }
                }
              ]
          }
       }
    }

    $ snoopit -s snoopers.json
     ERROR: Max sync retry count reached
     ERROR: Sync error on remote server syc_server_54
     ERROR: Switching to alternate server sync_server_702

Snooper Configuration

This is a JSON file which describes to the Snooper how to snoop around files and directories to find items of interest using regular expressions. It contains an array of files to snoop. Each file can be associated one or more regular expressions. Each regular expression behavior can be customized and each regular expression can be associated with zero or more event notifiers. The Snooper file also specifies notifier configurations and how to load custom notifiers.

    "snoopers" : {
                   "AppServer": {
                        "snoop": "/opt/servers/app_server/log/my_app_server.log"
                        "sniffers" : [
                            {
                                "comment" : "Bad status from server",
                                "regexp" : "Non OK Status",
                                "lines" : {
                                    "before" : 2,
                                    "after" : 2
                                },
                                "notify" : {
                                        "email" : {
                                            "to" : "admin@myplace.com"
                                        },
                                        "http" : {
                                            "url" : "http://localhost:3000/snoopers/snooped"
                                        }                                            
                                }
                            }
                         ]
                    }
    },
    "notifiers" :
      {
          "load" : {
              "MyNotifier" : {
                  "file" : "/home/joe_code/snoopit/my_notifier",
                  "class" : "MyNotifier",
                  "config" : { "config_param" : "config_value" }
              }
          },
          "email" : {
              "smtp-server" : "smtp.gmail.com",
              "port" : 587,
              "tls" : true,
              "user" : "someone@gmail.com",
              "password" : "apassword",
              "authentication" : "login"
          }
      }

Snoopers

Each Snooper configured in the JSON file is associated with either a file or a directory that will be snooped.

File Snoopers

Each file Snooper can be associated with one file.

    "snoopers": {
        "AppServer":  {
            "snoop": "/opt/servers/app_server/log/my_app_server.log"
        }
     }

The above specification will snoop the file /opt/servers/app_server/log/my_app_server.log

Directory Snoopers

Each directory Snooper can be associated with one directory. If the glob is not specified then every file in the directory is snooped. The glob string value is passed to Ruby's Dir.glob

    "snoopers": {
        "AppServer": {
              "dir" : {
                  "path" : "/opt/servers/app_server/log",
                  "glob" : "*.log"
                }
         }
    }

The above specification will snoop all files in directory /opt/servers/app_server/log with a suffix of .log

Defining Snooper Regular Expressions

Each Snooper has one or more regular expression specifications. This array of regular expressions are used to sniff through the files. These are identified as Sniffers.

Sniffer Attributes

comment Typically used by a Notifier, such as in the subject line in an email notifier.
regexp The value of this string is passed to ruby's Regexp
lines
- before Number of lines to print out before the matched line
- after Number of lines to print out after the matched line

notify This is a list of event notifiers to use when a line is matched by the regexp

"sniffers" : [
    {
        "comment" : "Bad status from server",
        "regexp" : "Non OK Status",
        "lines" : {
            "before" : 2,
            "after" : 2
        },
        "notify" : {
                "email" : {
                    "to": [ "watcher@something.com", "admin@something.com" ],
                    "from" : "snooper@something.com"
                }
        }
    }
 ]

Notifiers

After a file has been snooped by a Snooper any items matched by a Sniffer's regular expression can be sent to a specified notifier. Notifier's can be used to send notifications, such as an email, text messages, enqueue into a queueing system, or call a another service to take action based on the matched event.

Using a Notifier

Each Sniffer can use one or more notifiers. Notification parameters are specified for each Sniffer that uses a particular notifier. In the example above the email notifier parameters used are the to and from parameters.

Configuring a Notifier for the Manager

Notifiers will most likely need some minimal configuration information. These notifier configurations are specified after the snoopers section of the Snooper configuration file.

   "notifiers" :
        {
            "email" : {
                "smtp-server" : "smtp.gmail.com",
                "port" : 587,
                "user" : "someone@gmail.com",
                "password" : "password",
                "authentication" : "login"
            }
        }

Email Notifier

The email notifier is a simple SMTP mail notification provider.

Email Notifier Sniffer Parameters

The following parameters can be used with the email notifier

to Specifies who is to receive the email notification information. This can be a single string or an array
```
"foo@bar.com" or [ "foo@bar.com", "bar@foo.com" ]
```

Email Notifier Configuration

The following configuration information is specified in the notifiers section of the Snooper configuration file.

email Section name
smtp-server SMTP sever name
port SMTP port
user User name
password Password
authentication Authentication type can be one of the following plain,login,cram_md5

Stomp Notifier

The stomp notifier is a simple Stomp notification provider.

Stomp Notifier Sniffer Parameters

The following parameters can be used with the stomp notifier

The destination queue and `headers can be specified.

"queue" : "/queue/snooped",
"headers" : {
    "key-a": "a",
    "key-b": "b"
}

Stomp Notifier Configuration

The following configuration information is specified in the notifiers section of the Snooper configuration file.

"stomp": {
    "host": "localhost",
    "port": "61613",
    "login": "admin",
    "passcode": "flintstone",
    "headers": {
        "accept-version": "1.1",
        "host": "vhost"
    }
}

Redis Notifier

Available soon.

HTTP Post Notifier

The http notifier is a simple HTTP notification provider. It can handle post http and https. Basic authentication is available.

HTTP Post Notifier Sniffer Parameters

The following parameters can be used with the http notifier

url Specifies where to post the notification information.

  "http" : {
     "url" : "http://localhost:3000/snoopers/snooped"
  }

or for https

  "https" : {
     "url" : "https://localhost:3000/snoopers/snooped"
  }

HTTP Post Notifier Configuration

The following configuration information is specified in the notifiers section of the Snooper configuration file.

http Section name. Use http not https
api_key The value of this parameter is placed in the header with the following format
```
Authorization: Token token=BR549
```
user If a user parameter is passed then basic authentication will be performed with the given password
password If a password parameter is passed then basic authentication will be performed with the given user name
```
"http" : {
    "api-key" : "BR549",
    "user" : "fred",
    "password" : "flintstone"
}
```

Building Custom Notifiers

All Notifiers must inherit from the class Snoopit::Notifier and implement the notify method

  def notify(found, notify_params)
    raise NotImplementedError.new 'Notifier#notify'
  end

`notify` Parameters

found is a hash with the matched regular expression. It's what the Snooper sniffed out from the file.

  {
      comment: comment,
      file: file,
      before: [ lines before match ],
      match: matched_line,
      match_line_no: line_no_of_match,
      after: [ lines after match ]
  }

notify_params These are the parameters defined for the notifier in the Sniffer notifier section (e.g. to and from parameters for the email notifier)

"sniffers" : [
    {
        "comment" : "Bad status from server",
        "regexp" : "Non OK Status",
        "lines" : {
            "before" : 2,
            "after" : 2
        },
        "notify" : {
                "My Custom Notifier" : {
                    "param1": [ "a_value1", "a_value2" ],
                    "param2" : "value1"
                }
        }
    }
 ]

Dynamically Loading Custom Notifiers

To make a non default notifier available to the Snooper a notifier must be dynamically loaded. Specifying a notifier to be dynamically loaded is specified in the notifiers section of the Snooper configuration file.

    "notifiers" {
                    "load" : {
                        "My Custom Notifier" : {
                            "file" : "/opt/snooper/notifiers/my_custom_notifier",
                            "class" : "MyCustomNotifier",
                            "config" : { "param1": "configure_me" }
                        }
                    }
                }

In the notifiers Hash in the load Hash add the following in Custom Notifier Hash

A key with a descriptive unique name My Custom Notifier
file The path to the ruby file that contains the custom notifier class "/opt/snooper/notifiers/my_custom_notifier"
class The class name of your notifier MyCustomNotifier
config Any parameters needed to configure the custom notifier prior to generating notifications

Custom Notifier Initialization

Internally the Snooper will use an initializer with no arguments to create a custom notifier

MyCustomNotifier.new specified_config_params

If there are config items specified then the initializer will pass those items to the Custom Notifier#initializer

def initializer(config)
  super config
  @my_param1 = config['param1']
  @my_param2 = @config['param2'] 
  ...

Be sure to call super config in your initializer. One of the functions of calling super is to place the config parameter into the instance variable @config

Specifying Custom Notifier Configuration Section Names

The key used for your notification parameters for both configuring and notifying is the class name:

MyCustomNotifier.class.name

This can be changed in the MyCustomNotifier#initializer by passing a string to super

def initializer(config)
  super config, 'my_custom_notifier_name'
  @my_param1 = config['param1']
  @my_param2 = @config['param2'] 
  ...

File Tracking

Typically the Snooper is used for repeated invocations on a log file. This is typically done via cron. Enabling file tracking allows the Snooper to keep track of where it was in a file on it's last snoop of the file. Enabling file tracking prevents rereading the whole file and resending matched events. When file tracking is enabled using by the -T to snoopit the Snooper creates a JSON database in the directory where the Snooper was invoked. This file has the name snoopit_db.json The location of this file can be changed using the -f option of snoopit. If the -f option is used the -T is implied and does not need to be specified.

Cron example

*/15 * * * * /usr/local/bin/ruby /usr/local/ruby/gems/bin/snoopit -s /opt/admin/snoopit/cron_snoopit.json -f /opt/admin/snoopit/cron_snoopit_db.json -n >> /opt/admin/snoopit/cron_snoopit.out 2>&1

History

Written this handy little utility too many times too count. From the ancient times via Rob Pikes and the gang's sh,grep, awk, sed and mail to Larry Wall's wonderful perl to the latest and best yet ruby from Matz.

    grep -H -n -B 2 -A 2 'Look for this' ./log/some.log | awk ... |  mail ...

Versioning

This library aims to adhere to Semantic Versioning 2.0.0. Violations of this scheme should be reported as bugs. Specifically, if a minor or patch version is released that breaks backward compatibility, that version should be immediately yanked and/or a new version should be immediately released that restores compatibility. Breaking changes to the public API will only be introduced with new major versions. As a result of this policy, you can (and should) specify a dependency on this gem using the Pessimistic Version Constraint with two digits of precision.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

Snoopit

Simple tool for monitoring process log files for specified events and then generating a basic notification.

Snoopit

Installation

Usage

Basic Snoopers specification file

Snooper Configuration

Snoopers

File Snoopers

Directory Snoopers

Defining Snooper Regular Expressions

Sniffer Attributes

Notifiers

Using a Notifier

Configuring a Notifier for the Manager

Email Notifier

Email Notifier Sniffer Parameters

Email Notifier Configuration

Stomp Notifier

Stomp Notifier Sniffer Parameters

Stomp Notifier Configuration

Redis Notifier

HTTP Post Notifier

HTTP Post Notifier Sniffer Parameters

HTTP Post Notifier Configuration

Building Custom Notifiers

notify Parameters

Dynamically Loading Custom Notifiers

Custom Notifier Initialization

Specifying Custom Notifier Configuration Section Names

File Tracking

Cron example

History

Versioning

Contributing

`notify` Parameters