Solaris 10: how to install and configure a SNMP trap translator daemon


We started this journey about SNMP traps installing a SNMP Trap Handler, which is part of Net-SNMP package, in my previous post. Now, the last tile of this processing chain is a SNMP Trap Translator in charge of “translating” notifications received from snmptrapd daemon into easy to understand messages by means variable substitution. In addition, SNMPTT can also filter traps using their attributes (hostname, ipa, etc.), apply regexp search & replace algorithms, log to several kind of destinations (syslog-ng as well) or pass translated messages to monitoring clients (like Nagios) by executing external programs or scripts. Yes, it’s a full featured snmp tool indeed! Now, let’s see how to make it available to the end user.

Schema

SNMP Trap

Architecture

The SNMP Trap Translator is a structure based on the following components:

  • /usr/local/sbin/snmptt: the daemon, it’s a Perl script responsible for translating received traps on a customized format;
  • /etc/init.d/snmptt: a launch script which allow to start and stop the daemon easily;
  • /etc/snmp/conf/snmptt.ini: the daemon configuration file;
  • /etc/snmp/snmptt.conf: a configuration file including all MIBs definitions and their translations rules;
  • /var/spool/snmptt: temporary store for received traps.
  • /var/log/snmptt: log files directory;

Pre-requisites

Perl is your first requirement and in this case you have the choice between using an already built package or (more challenging!) compiling it by yourself using the instructions reported in my post. You decide! 😉

Furthermore, before using the SNMP Trap Translator Perl scripts, you need some additional modules that you can download from CPAN (if you’re in trouble use my post about this topic):

  • Exporter::Tiny
  • List::MoreUtils
  • Config::IniFiles

Other required modules that should be already part of your Perl distribution are:

  • Text::ParseWords
  • Getopt::Long
  • POSIX
  • Time::HiRes
  • Sys::Hostname
  • File::Basename
  • Text::Balanced

Install

Download the SNMP Trap Translator package from its homesite.

The installation process should be carried out with the “root” user:

# su - root

Let’s start by unzipping snmptt’s source files under /tmp directory (take care to use the GNU version of the tar command!), then make “root” their owner and group:

# cd /tmp
# /usr/local/bin/tar xzvf snmptt_1.4.tgz
# chown -R root:root snmptt_1.4
# rm snmptt_1.4.tgz

Then, make a copy of snmptt, snmptthandler and snmpttconvertmib files under /usr/local/sbin, giving all execution permissions:

# cd snmptt_1.4
# cp snmptt snmptthandler snmpttconvertmib /usr/local/sbin
# chmod +x /usr/local/sbin/snmptt
# chmod +x /usr/local/sbin/snmptthandler
# chmod +x /usr/local/sbin/snmpttconvertmib

It’s also mandatory to modify all the above scripts, so that they point to the right version of Perl (in my case is /usr/local/bin/perl):

# vi /usr/sbin/snmptt /usr/sbin/snmptthandler /usr/sbin/snmpttconvertmib
  FROM : #!/usr/bin/perl
  TO   : #!/usr/local/bin/perl

After that, copy snmptt.ini configuration file under /etc/snmp/conf and change its owner and group:

# cp snmptt.ini /etc/snmp/conf/
# chown root:sys /etc/snmp/conf/snmptt.ini

You can also exploit the bundled snmptt.logrotate file if you have logrotate installed in your operating system (see my post). Since we are going to use the Service Management Facility to manage the SNMPTT daemon, please change the logrotate references to the “snmptt” script with the invocation of the svcadm command, as shown in the file below:


/var/log/snmptt/snmptt*.log /var/log/snmptt/snmptthandler.debug {
    weekly
    notifempty
    missingok
}

/var/log/snmptt/snmptt.debug {
    weekly
    notifempty
    missingok
    postrotate
        /usr/sbin/svcadm restart snmptt >/dev/null 2>/dev/null || true
    endscript
}

Copy it in the common configuration path /etc/logrotate.d and schedule an execution in root’s crontab:

# cp snmptt.logrotate /etc/logrotate.d/snmptt
# export EDITOR=vi
# crontab -e
  00 00 * * * /usr/local/sbin/logrotate /etc/logrotate.d/snmptt > /dev/null 2>&1

Then, we need to create the following paths:

  • a spool directory to read received traps from:
  • # mkdir /var/spool/snmptt
    
  • a log directory where traps processing is described:
  • # mkdir /var/log/snmptt
    

Finally, remove source directory:

# rm -R /tmp/snmptt_1.4

Configure

SNMP Trap Translator has a unique file for configuration, named snmptt.ini. Everyone has its own requirements, so it’s quite difficult to define a general configuration good for every situation. With respect to my needs, I usually change the following default values:

  • General ⇒ mode = daemon
  • General ⇒ multiple_event = 0
  • General ⇒ description_mode = 1 (means enable trap description text taken from MIBs)
  • DaemonMode ⇒ daemon_uid = blank (means no child process with different UID)
  • Logging ⇒ log_system_enable = 1
  • Logging ⇒ unknown_trap_log_enable = 1
  • Logging ⇒ syslog_enable = 0
  • Logging ⇒ syslog_facility = daemon
  • Logging ⇒ syslog_system_facility = daemon
  • Debugging ⇒ DEBUGGING = 2 (means out all messages)
  • Debugging ⇒ DEBUGGING_FILE, uncomment already defined path
  • Debugging ⇒ DEBUGGING_FILE_HANDLER, uncomment already defined path

Regarding the “multiple_event” field, I don’t want multiple trap definitions to be executed for the same trap notification, so I prefer to stop after the first match.

If you have a domain and want to strip it from the host name that send the trap, you can define your domain names and activate this feature:

  • General ⇒ strip_domain = 2 (means strip domains define by the list)
  • General ⇒ strip_domain_list = “your-domain

Finally, at the end of the file you can specify a list of configuration files describing what events have to be recognized and, for each one, the action to be executed. Read the convertmib doc page for further informations about how a configuration file of this kind can be created. The action taken at the SNMP Trap arrival can be whatever, for example a common used task is to write a message on a file watched by a monitoring system, for example Nagios.

Start SNMP trap translator as a service (using SMF)

Login as “root” user and copy this customized startup / shutdown script:


#!/bin/sh
#
# $Id: init.d.solaris,v 1.0 2009/11/06 08:00:00 Luca Merello Exp $
#

###########
# PROGRAM #
###########

PROG_DESCR=SNMPTT
PROG_NAME=/usr/local/sbin/snmptt
PROG_PID=/var/run/snmptt.pid
PROG_OPTIONS="--daemon --ini=/etc/snmp/conf/snmptt.ini"

#############
# FUNCTIONS #
#############

RETVAL=0

start()
{
	echo -n $"Starting $PROG_DESCR ..."
        $PROG_NAME $PROG_OPTIONS
	RETVAL=$?
	echo
	return $RETVAL
}

stop()
{
	echo -n $"Stopping $PROG_DESCR ..."
	pkill -f $PROG_NAME 2>/dev/null
	RETVAL=$?
	echo
	if test -f $PROG_PID ; then
	  [ $RETVAL -eq 0 ] && rm -f $PROG_PID
	fi
	return $RETVAL
}

reload()
{
    echo -n $"Reloading config file of $PROG_DESCR ..."
    pkill -HUP -f $PROG_NAME
    RETVAL=$?
    echo
    return $RETVAL
}

restart()
{
	stop
	start
}

case "$1" in
  start)
	start
	;;
  stop)
	stop
	;;
  restart)
	restart
        ;;
  reload|force-reload)
	reload
        ;;
  *)
	echo $"Usage: $0 {start|stop|restart|reload|force-reload}"
	RETVAL=1
esac

exit $RETVAL

to /etc/init.d directory:

# su - root
# mv /tmp/snmptt /etc/init.d/
# chown root:sys /etc/init.d/snmptt
# chmod 744 /etc/init.d/snmptt

After that, copy this customized manifest file:


<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='snmptt'>

<service name='application/snmp/snmptt' type='service' version='1'>

    <create_default_instance enabled='false' />

    <single_instance />

    <!--
        Wait for network interfaces to be initialized.
    -->
    <dependency
        name='network'
        grouping='require_all'
        restart_on='none'
        type='service'>
        <service_fmri value='svc:/milestone/network:default' />
    </dependency>

    <!--
        Wait for all local filesystems to be mounted.
    -->
    <dependency
        name='filesystem-local'
        grouping='require_all'
        restart_on='none'
        type='service'>
        <service_fmri value='svc:/system/filesystem/local:default' />
    </dependency>


    <exec_method
        type='method'
        name='start'
        exec='/etc/init.d/snmptt start'
        timeout_seconds='60' />

    <exec_method
        type='method'
        name='stop'
        exec='/etc/init.d/snmptt stop'
        timeout_seconds='60' />

    <exec_method
        type='method'
        name='restart'
        exec='/etc/init.d/snmptt restart'
        timeout_seconds='60' />

    <stability value='Evolving' />

    <template>
        <common_name>
            <loctext xml:lang='C'>
                SNMP Trap Translator
            </loctext>
        </common_name>
    </template>

</service>

</service_bundle>

to /tmp directory, then enable the snmptt service through Solaris SMF with the following instructions:

# mv /tmp/snmptt.xml /var/svc/manifest/application/snmp/
# chown root:sys /var/svc/manifest/application/snmp/snmptt.xml
# chmod 444 /var/svc/manifest/application/snmp/snmptt.xml
# svccfg validate /var/svc/manifest/application/snmp/snmptt.xml
# svccfg -v import /var/svc/manifest/application/snmp/snmptt.xml
  svccfg: Taking "initial" snapshot for svc:/application/snmp/snmptt:default.
  svccfg: Taking "last-import" snapshot for svc:/application/snmp/snmptt:default.
  svccfg: Refreshed svc:/application/snmp/snmptt:default.
  svccfg: Successful import.

Note:
if something goes wrong during manifest import, the procedure can be repeated after deleting the old file as follows:

# svccfg delete svc:/application/snmp/snmptt:default
# svcs -xv snmptt
  svc:/application/snmp/snmptt:default (SNMP Trap Translator)
  State: disabled since January 01, 2015 08:00:00 AM CET
  Reason: Disabled by an administrator.
  See: http://sun.com/msg/SMF-8000-05
  Impact: This service is not running.
# svcadm enable snmptt

Then, check corresponding log file so as to verify a proper start:

# more /var/svc/log/application-snmp-snmptt:default.log
  [ Jan 01 08:00:00 Enabled. ]
  [ Jan 01 08:00:00 Executing start method ("/etc/init.d/snmptt start") ]
  -n $Starting SNMPTT ...
  Could not open configuration file: /etc/snmp/snmptt.conf (No such file or directory) at /usr/local/sbin/snmptt line 2723.
  PID file: /var/run/snmptt.pid
  [ Jan 01 08:00:00 Method "start" exited with status 0 ]

Note that the lack of configuration file /etc/snmp/snmptt.conf is expected, since at this time we have not yet converted any MIB for the translating process (see configuration paragraph).

If you want to go into details, use ps command to see the daemon running:

# ps -ef | grep snmptt
  root 28035     1   0   Jan 01 ?    0:03 /usr/local/bin/perl /usr/local/sbin/snmptt --daemon --ini=/etc/snmp/conf/snmptt

In this way, starting, stopping and restarting snmptt can be accomplished exclusively using svcadm command:

# svcadm enable  snmptt
# svcadm disable snmptt
# svcadm restart snmptt


Even this effort was completed at the end … make good use! 😀