PKG-SNF-CS-NIX/SNF_CS_Developer_Package/INSTALL

               SNFServer Installation and Configuration

              Copyright (C) 2008 ARM Research Labs, LLC.

           See www.armresearch.com for the copyright terms.
                                   
Installing SNFServer to filter email involves the following steps:

  1) Check prerequisites.

  2) Create the snfuser user and group.

  3) Build and install the SNFServer package (using a tarball or a
  package).

  4) Configure the SNFServer package.

  5) Interface with the MTA (postfix or sendmail).

  6) Configure the OS to start and stop SNFServer on bootup and
  shutdown.

The following sections describes each of these steps for a default
installation.  Any OS-specific issues are described at the end of each
section.

Prerequisites
*************

Before installing SNFServer, make sure that:

  1) The program curl must be installed.

Creating the snfuser user and group
***********************************

Before installing, the snfuser user and group must be created.  For
increased security, snfuser user has no shell.

OS-specific issues--

The commands to create the snfuser user and group are OS dependent.
For your convenience, the commands for creating the user and group for
varous OSes are listed here.  However, no guarantee is made that these
commands will work on your system; please refer to your system
documentation.

  1) OpenBSD:

     a) 'useradd -g =uid -m -c "Sniffer Account" -s /bin/false snfuser'.

  2) Ubuntu:

     a) 'adduser --gecos "Sniffer Account" --no-create-home --shell /bin/false snfuser'.

  3) RedHat (and variants such as Fedora and CentOS):

     a) 'adduser --comment "Sniffer Account" -M --shell /bin/false snfuser'

  4) Suse:

     a) 'groupadd snfuser'

     b) 'useradd -c "Sniffer Account" -s /bin/false -g snfuser snfuser'

  5) FreeBSD:

     a) 'pw user add -c "Sniffer Account" -n snfuser -w no -s /bin/false'

Building and installing SNFServer
*********************************

For some operating systems, configuration files are normally installed
in /etc, and the installation of the remaining files are in /usr
(i.e. /usr/sbin, /usr/share/snf-server, etc).  In other operating
systems, the files are installed in /usr/local/etc or /usr/local.

The directories to install are specified by the '--prefix' and
'--sysconfdir' options to the configure script (see below).

To build and install from the tarball--

  1) Extract the distribution from the tarball: 'tar xvzf
  snf-server-X.Y.Z.tar.gz', where X.Y.Z is the version of the
  distribution.

  1) 'cd' to the source directory.

  2) Run the configure script.  See the section on OS-specific issues
  for the options to specify.  This configures the package.

  3) Type 'make'.  This builds the package.

  4) Type 'make install'.  This installs the package in the default
  directories.

The following files are installed:

  Executables and scripts in sbin:
       SNFServer
       SNFClient
       SNF2Check
       getRulebase.sample
       snfSniffer.sample
       snfSnifferFilter.sample
       snfscan-standalone.sample
       snfServerControl.sample

  Sample configuration files in etc/snf-server:
       SNFServer.xml.sample
       identity.xml.sample

  In share/snf-server
       GBUdbIgnoreList.txt.sample

During operation of SNFServer, the following files are created:

  In share/snf-server:
       *.snf
       Log files
       UpdateReady.txt
       Temporary files

OS-specific options for ./configure:

  1) OpenBSD:

    ./configure --enable-os-type=OpenBSD --sysconfdir=/etc

  2) FreeBSD:

    ./configure --enable-os-type=FreeBSD

  3) Ubuntu:

    ./configure --enable-os-type=Ubuntu --sysconfdir=/etc --prefix=/usr

  4) RedHat (and variants such as Fedora and CentOS):

    ./configure --enable-os-type=RedHat --sysconfdir=/etc --prefix=/usr

  5) Suse:

    ./configure --enable-os-type=Suse --sysconfdir=/etc --prefix=/usr

Configuration
*************

Configuration consists of creating the configuration files used by
SNFServer from the sample configuration files of the distribution.
There are three configuration files.  The location of the files
described in this section are for the installation in /usr and /etc.

Two of the files are in /etc/snf-server:

  1) SNFServer.xml (sample configuration file is
  SNFServer.xml.sample).

  2) identity.xml (sample configuration file is identity.xml.sample).

and one in /usr/share/snf-server:

  1) GBUdbIngoreList.txt (sample configuration file is
  GBUdbIgnoreList.txt.sample).

To configure SNFServer, do the following:

  1) Copy SNFServer.xml.sample to SNFServer.xml, and edit as follows:

     a) Specify the directories.  The default directories are:

          <node identity='/etc/snf-server/identity.xml'>
          <log path='/var/log/snf-server/'/>
          <rulebase path='/usr/share/snf-server/'/>
          <workspace path='/usr/share/snf-server/'/>
          <update-script on-off='on' call='/usr/sbin/getRulebase'
                         guard-time='180'/>

     The paths need to be changed only if the default directories are
     not used.

     If desired for security purposes, restrict the permissions of
     SNFServer.xml.  For example, to make SNFServer.xml readonly by
     only the snfuser user and snfuser group, enter the following:

       chmod 440 SNFServer.xml

  2) Copy identity.xml.sample to identity.xml, and edit to include the
  license ID and authentication attributes of the <identity> element.
  If desired for security purposes, restrict the permissions of
  identity.xml.  For example, to make identity.xml readonly by only
  snfuser, enter the following:

    chmod 400 identity.xml

  3) Copy GBUdbIngoreList.txt.sample to GBUdbIgnoreList.txt, and edit
  as appropriate (the file format is described in the comments in the
  file).  See README for more information on this file.

  4) In the directory for the update script specified in the
  configuration file (default: /usr/sbin):

     a) Copy getRulebase.sample to getRulebase, and edit as follows:

       i) In the line

            AUTHENTICATION=authenticationxx

          replace authenticationxx with the authentication for the
          SNFServer license.
          
       ii) In the line

            LICENSE_ID=licenseid

          replace licenseid with the license ID of the SNFServer
          license.

       iii) Any other changes as necessary if the default directories
       are not used.

     b) Ensure that getRulebase is executable by the snfuser user.
     This can be done with the command:

       chmod +x getRulebase

  5) Ensure that the snfuser user has read/write access to the files
  in workspace (default: /usr/share/snf-server or
  /usr/local/share/snf-server) and configuration directory (default:
  /etc/snf-server).  To grant this access, enter the following
  command, as the root user:

    chown -R snfuser:snfuser /usr/share/snf-server

    chown -R snfuser:snfuser /etc/snf-server

  As you modify files in these directories, please ensure that the
  read/write permissions for snfuser is maintained.

  6) Create the logfile directory, and ensure the snfuser user has
  read/write access to it:

    mkdir /var/log/snf-server

    chown snfuser:snfuser /var/log/snf-server

    chmod 755 /var/log/snf-server

  7) Download the rulebase file:

     a) 'cd /usr/share/snf-server'.  If the workspace specified in the
     configuration file is not the default, this command should be
     changed accordingly.

     b) 'touch UpdateReady.txt'.

     c) 'chown snfuser UpdateReady.txt'.

     d) 'su -m snfuser -c "/usr/sbin/getRulebase"'.  If getRulebase is
     in a different directory, this command should be changed
     accordingly.

OS-specific issues--

  None.

Configuring the OS
******************

The OS can be configured to automatically start and stop SNFServer on
system startup and shutdown.  The following gives the procedure for
each OS:

OpenBSD:

  1) Add the following line to /etc/rc.local:

       /usr/local/sbin/snf-server start

  2) Add the following line to /etc/rc.shutdown:

       /usr/local/sbin/snf-server stop

  3) Run '/usr/local/sbin/snf-server start' to start the server.

FreeBSD:

  1) Create the directory /etc/rc.conf.d (if it doesn't exist).

  2) Create the file /etc/rc.conf.d/snfserver with the contents:

       snfserver_enable="YES"

  3) Run '/usr/local/etc/rc.d/snf-server start' to start the server.

Ubuntu:

  1) Set the VERBOSE variable in /etc/default/rcS to control the
  script output.

  2) Configure the system to run the script.  With sysv-rc-conf:

       sysv-rc-conf -level 2345 snf-server on

     will start the server for runlevels 2, 3, 4, and 5.

     And

       sysv-rc-conf --list snf-server

     will list the runlevels for which the server is started.

  3) Run '/etc/init.d/snf-server start' to start the server.

RedHat (and variants such as Fedora) and Suse:

  1) Run 'chkconfig --add snf-server'.  This sonfigures the system to
  run the script on startup and shutdown.

  2) Run 'chkconfig --list snf-server' to display the runlevels

  3) Run 'service snf-server start' to start the SNFServer.

Integration with MTAs
*********************

The section assumes you will be using SNFServer with an MTA and simply
injecting headers that will be used later to remove, quarantine, or
otherwise redirect messages detected as spam. There are as many ways
to use SNFServer as there are systems using it -- so the following is
just a good starting point.

It is presumed that SNFServer is configured with x-header injection
turned on and that the x-headers have been customized to suit your
needs. Check the <xheaders/> section of your
/etc/snf-server/SNFServer.xml file to verify that SNFServer is
configured to do what you want.

Integration with postfix
------------------------

One way to integrate with postfix is to configure postfix to use the
snfSniffer script (described below) as a content filter.  The
snfSniffer script passes the message through SNFServer, and then
reinjects the message into the mail system.

  1) In /usr/sbin (or /usr/local/sbin), copy the script
  snfSniffer.sample to snfSniffer set the correct access rights:

    cp snfSniffer.sample snfSniffer

    chown snfuser snfSniffer

    chmod 550 snfSniffer

The default snfSniffer script creates a temporary copy of the message,
scans it with SNFServer, and then reinjects the message.

   2) Change /etc/postfix/master.cf as follows (LEADING WHITE SPACES
   ARE IMPORTANT WHEN MAKING THIS CHANGE):

   change (each line is enclosed in single quotes; add only the text
   between the single quotes):

      'smtp      inet  n       -       n       -       -       smtpd'

   to:

       'smtp      inet  n       -       n       -       -       smtpd'
       '     -o content_filter=snfilter:dummy'


    also add:

       'snfilter  unix  -       n       n       -       10      pipe'
       '     flags=Rq user=snfuser argv=/usr/sbin/snfSniffer'
       '     -f ${sender} -- ${recipient}'

    to master.cf.  Specify the directory snfSniffer is in if not
    /usr/sbin.

At this point you could just restart postfix, and hope nothing goes
wrong. Instead, it would be smarter to first test the installation
from the command line by injecting a message directly into the filter
script "snfSniffer".  We can issue a command like (in directory
/usr/share/snf-server):

  /usr/sbin/snfSniffer -f sender recipient < junkmsg.txt

Where junkmsg.txt is a spam test message. We should also test
a clean message to make sure that this script is working as we
expect it to.  In this case we would issue a command like

  /usr/sbin/snfSniffer -f sender recipient < cleanmsg.txt

If you've done everything correctly then all you have to do is reload
postfix to start the content_filter working.

   4) 'postfix reload'.  Restart postfix.

   5) To stop using the snfSniffer filter:

      i) Comment out or remove the '-o content_filter=snfilter:dummy'
      line in the /etc/postfix/master.cf file.

      ii) 'postsuper -r ALL' to remove content filter request records
      from existing queue files.

      iii) 'postfix reload' to reload the postfix configuration.

OS-specific issues--

  1) OpenBSD:  In the default postfix configuration for OpenBSD, the
  postfix programs run chrooted in /var/spool/postfix.  For that
  configuration, the /etc/pwd.db file needs to be made available to
  the postfix programs.  This can be done by copying pwd.db:

    cp /etc/pwd.db /var/spool/postfix/etc

  If the /etc/pwd.db file is modified after copying, the postfix
  system issues a warning.  To avoid this warning, copy the
  /etc/pwd.db file to /var/spool/postfix/etc directory whenever
  /etc/pwd.db is modified.

Integration with sendmail
-------------------------

One way to integrate with send is to configure sendmail to use the
snfSnifferFilter script (described below) as a filter.  The snfSniffer
script passes the message through SNFServer, and then sends the
message to stdout.  The integration is done by having sendmail use
procmail to filter all mail (which is normally the default), and
configuring procmail to use snfSnifferFilter as a filter.

  1) In /usr/sbin (or /usr/local/sbin), copy the script
  snfSnifferFilter.sample to snfSnifferFilter set the correct access
  rights:

    cp snfSnifferFilter.sample snfSnifferFilter

    chown snfuser snfSnifferFilter

    chmod 550 snfSnifferFilter

The default snfSnifferFilter script creates a temporary copy of the
message, scans it with SNFServer, and then send the message to
stdout.

   2) Verify that sendmail is configured to use procmail to deliver
   mail.  Please see the sendmail and procmail documentation for how
   to do this.

   3) Create the file /etc/procmailrc (or whatever the system procmail
   rcfile is; see the procmail documentation for the path on your
   system) to contain the following lines:

     ':0 fw'
     '| /usr/sbin/snfSnifferFilter'

   where the single quotes are not to be included.

At this point mail should be filtered by SNFServer.  To check this,
issue a command like (in directory /usr/share/snf-server):

  /usr/sbin/snfSnifferFilter < junkmsg.txt

Where junkmsg.txt is a spam test message.  We should also test a clean
message to make sure that this script is working as we expect it to.
In this case we would issue a command like

  /usr/sbin/snfSnifferFilter < cleanmsg.txt

In either case you should see the message with the headers added by
SNFServer.

   4) To stop using the snfSnifferFilter:

      i) Comment out or remove the two lines you added to /etc/procmailrc.

OS-specific issues--

  1) OpenBSD: The default sendmail confiugration might not use
  procmail.  To configure sendmail to use procmail, please ensure that
  procmail is installed and see the procmail documentation
  (/usr/local/share/examples/procmail/advanced) for the procedure to
  integrate procmail as an integrated local mail delivery agent.

  By default, procmail is installed in /usr/local/bin, and the system
  procmail rcfile is /etc/procmailrc.

  After you make the changes, stop sendmail with 'pkill sendmail', and
  start it with 'sendmail -bd'.

  2) FreeBSD: The default sendmail confiugration might not use
  procmail.  To configure sendmail to use procmail, please ensure that
  procmail is installed and see the procmail documentation
  (/usr/local/share/examples/procmail/advanced) for the procedure to
  integrate procmail as an integrated local mail delivery agent.

  By default, procmail is installed in /usr/local/bin, and the system
  procmail rcfile is /usr/local/etc/procmailrc.

  After you make the changes, restart sendmail with
  '/etc/rc.d/sendmail restart'.

  3) RedHat, Fedora, Centos, Suse, and Ubuntu: Restart sendmail with
  '/etc/init.d/sendmail restart'.

Debugging
---------

To obtain additional information for debugging, SNFServer can be run
in debug mode.  In debug mode, additional messages are output to a
file, and core dumps are enabled.  Debug mode is enabled and disabled
using the snf-server startup script.  Please see the section
"Configuring the OS" the the location of the startup script for each
OS.

To enable debug mode, run:

  snf-server debug_mode

Note: Running this command creates a file
/usr/share/snf-server/debug_mode that indicates to the snf-server
startup script to start SNFServer in debug mode.  This file contains
brief documentation about running in debug mode.

To enable production mode (non-debug mode), run:

  snf-server production_mode

Note: Running this command creates a file
/usr/share/snf-server/production_mode that indicates to the snf-server
startup script to start SNFServer in production mode.  This file has
the same content as /usr/share/snf-server/debug_mode.

After the desired mode is enabled, you can start or restart SNFServer
using the standard procedure.

In debug mode for all OSes except OpenBSD, SNFDebugServer is run with
strace (with the '-r -tt -v' options), and all the output is written
to /var/log/snf-server/debug.log.  Also, any core dumps are written to
/usr/share/snf-server.

In debug mode for OpenBSD, SNFDebugServer is run with ktrace.  The
output of SNFDebugServer is written to /var/log/snf-server/debug.log,
and the output of strace is written to /var/log/snf-server/ktrace.out.

OS-specific issues--

  1) FreeBSD: You'll need to ensure that core dumps are enabled.  On
  FreeBSD 7.0, the command to do this is:

    sysctl kern.coredump=1

  You'll also need to ensure that /proc is mounted.  You can mount
  /proc with the following command:

    mount -t procfs proc /proc

  When running in debug mode, 'snf-server start' displays the PID of
  the strace process instead of the SNFDebugServer process.
  'snf-server status' displays the PID values correctly.

  2) OpenBSD:  Use kdump to view the kernal trace file:


       kdump /var/log/snf-server/ktrace.out