SNMP Trap Monitoring in nagios

Introduction

To monitor SNMP supported devices like switches, router, firewall, UPS, Thermometer we need not to check actively the status of device. SNMP supported device provides trap incase of status change as described in MIB file of that device.

 SNMP Trap monitoring

The scenario is about nagios acting as a "manager" or (part of) a Network Management System (NMS), receiving messages from "Trap Agents" - conceptually similar to NSCA agents sending msgs to the NSCA Daemon with interacts with a Nagios passive service (add links), except that the values to be checked and the thresholds that trigger a message are handled with snmp tools.

Installation

To accept and process all traps from snmp supported devices we need to install few packages.

Following is the list of packages:

• snmpd
• snmp
• snmptt
• libnet-snmp-perl
• libsnmp-perl
• libsnmp-python
• libconfig-inifiles-perl
• libconfig-inifiles-perl

Installation methods:
All above mentioned packages are available in our repositories we can install it as follows

apt-get install snmpd snmp libnet-snmp-perl libsnmp-perl libsnmp-python libconfig-inifiles-perl libconfig-inifiles-perl snmptt

Now it is required to start snmpd service at boot time so we need to add it startup 

insserv snmpd

Configuration

[ Note :  * In configuration part you need to remove those parameters which written with RED color and add  those parameters which written with GREEN colour. ]

vim /etc/snmp/snmpd.conf

• agentAddress udp:127.0.0.1:161                     ###     (remove this line )
•  agentAddress  udp:192.168.1.66:161              ###    (add this line)

In  /etc/defaults/snmp update following parameters. 
==>Default TRAPD is disabled

• TRAPDRUN=no  

 Which is required to enable So enable it by changing "no" to "yes"

• TRAPDRUN=yes  
• TRAPDOPTS='-n -Lf /var/log/snmptrapd.log -p /var/run/snmptrapd.pid'

==>Default SNMPD is enabled

• SNMPDRUN=yes

Which is not required to enable So disable it by changing "yes" to "no" 

• SNMPDRUN=no

In  /etc/snmp/snmptrapd.conf  update following parameters. Add following lines.

• disableAuthorization yes
•  traphandle default /usr/sbin/snmptthandler

In  /etc/snmp/snmptt.ini  update following parameters.

•  mode = daemon
•  net_snmp_perl_enable = 1
•  net_snmp_perl_best_guess = 2
•  translate_log_trap_oid = 2
•  remove_backslash_from_quotes = 1
•  description_mode = 2
•  date_time_format = %Y-%m-%d %H:%M:%S
•  unknown_trap_log_enable = 1
•  DEBUGGING_FILE = /var/log/snmptt.debug
•  DEBUGGING = 0 

 Update follwoing perameteres if you want to configure smtp with eventdb

•  db_translate_enterprise = 1
•  db_unknown_trap_format = 'Unknown Trap: $-*'
•  mysql_dbi_enable = 1
•  mysql_dbi_host = <eventdb database host)
•  mysql_dbi_database = <eventdb database name>
•  mysql_dbi_table = <eventdb table name for known traps>
•  mysql_dbi_table_unknown = <eventdb table name for known traps>
•  mysql_dbi_username = <Username for eventdb database access>
•  mysql_dbi_password = < Password for eventdb database access>
•  date_time_format_sql = %Y-%m-%d %H:%M:%S
•  stat_time_format_sql = %Y-%m-%d %H:%M:%S

Upload Trap definition for devices

Generate trap definition from mibs for all devices by following command and add those file names in /etc/snmp/snmptt.ini

snmpttconvertmib --in=MIBFILE --out=/etc/snmp/snmptt.conf --exec='/usr/local/nagios/libexec/eventhandlers/submit_check_result $A TRAP <STATUS_NO_NAGIOS>' 

For Example:

snmpttconvertmib --in=MIBFILE --out=/etc/snmp/snmptt.conf --exec='/usr/local/nagios/libexec/eventhandlers/submit_check_result $A TRAP 2' 

$A is the hostipaddress can be found in received traps

TRAP is the service defined on host 

2 is the status of service which you can have in nagios

Create Service in NAGIOS

Now create service (like. TRAP) in NAGIOS with following parameters.

EXAMPLE : Service name TRAP 

define service{ 
name trap-service 
register 0 
service_description TRAP 
is_volatile 1 
check_command check-host-alive 
max_check_attempts 1 
normal_check_interval 1 
retry_check_interval 1 
passive_checks_enabled 1 
check_period never
notification_interval 0 
contact_groups admins
check_period 24x7 
max_check_attempts 3 
normal_check_interval 15 
retry_check_interval 5 
active_checks_enabled 1 
passive_checks_enabled 0 
parallelize_check 1 
obsess_over_service 0 
check_freshness 0 
event_handler_enabled 0 
flap_detection_enabled 0 
process_perf_data 1 
retain_status_information 1 
retain_nonstatus_information 1 
notification_interval 60 
notification_period 24x7 
notification_options w,u,c,r 
notifications_enabled 1 
}

###################################

SUBMIT_CHECK_RESULT Plugin For nagios

###################################

#!/bin/bash
########################################################################################################
#Description  : This script will write a command to the Nagios command file to cause Nagios to process a passive service check result. Note: This script is intended to be run on the same host that is running Nagios  Main host and it will send those traps on that host to nagios instances if the trap found for the device configured on any of the nagios instance.
#################################################################s########################################
##Change Log  :
# Arguments:
#  $1 = ipaddress of host that the service is associated with
#  $2 = svc_description (Description of the service)
#  $3 = return_code (An integer that determines the state of the service check, 0=OK, 1=WARNING, 2=CRITICAL, 3=UNKNOWN).
#  $4 = plugin_output (A text string that should be used as the plugin output for the service check)
#
#################################################################s########################################
#binary paths
check_nrpe="/usr/lib/nagios/plugins/check_nrpe"
echocmd="/bin/echo"
db_name="nagios"
db_user="root"
db_passwd="password"

#Nagios CommandFile in which command will be passed to nagios 
CommandFile="/var/lib/nagios/rw/nagios.cmd"

#gets nagios instance and host_name of the source of trap from the ipaddress
host_name=`/usr/bin/mysql --skip-column-names -u $db_user -p$db_passwd $db_name -e"select instance_id,display_name from nagios_hosts where address=\"$1\"" | /usr/bin/expand -t 1`
instance=`echo $host_name|cut -d' ' -f1`
host_name=`echo $host_name|cut -d' ' -f2`

# get the current date/time in seconds since UNIX epoch
datetime=`date +%s`

# check thwe instance and forward nagios command to respected nagios instance
if [ $instance -eq 1 ]
then
        cmdline="$host_name;$2;$3;$4"
#       append the command to the end of the command file
        echo "[$datetime] PROCESS_SERVICE_CHECK_RESULT;$cmdline" >> $CommandFile
else
        print "Unknown. Host not found."
fi