Posted on

Upgrading Logwatch on CentOS 5

Introduction

I finally got tired at looking at the thousand-plus line daily reports coming to my inbox from Logwatch every evening.  Don’t get me wrong, I love logwatch.  It helps me keep an eye on my servers without having to scrutinize every log file.  If you aren’t using logwatch on your Linux boxes I strongly suggest you look into it and turn on this very valuable service.  Most Linux distros come with this pre-installed.

The problem is that on CentOS the version of logwatch that comes with the system was last updated in 2006.   The logwatch project itself, however, was updated just a few months ago.  As of this writing the version running on CentOS 5 is 7.3 (released 03/24/06) and the version on the logwatch SourceForge site is 7.3.6 (updated March 2010).   In this latest version there are a log of nice updates to the scripts that monitor your log files for you.

The one I’m after, consolidating brute force hacking attempt reports, is a BIG thing.  We see thousands of entries in our daily log files from China hackers trying to get into our servers.   This is typical of most servers these days, however in many cases ignorance is bliss.  Many site owners and IPPs don’t have logging turned on because they get sick of all the reports of hacking attempts.  Luckily we block these attempts on our server, but our Fireant labs project is configured to have iptables tell us whenever an attempt is blocked at the kernel level (we like to monitor what our labs scripts are doing while they are still in alpha testing).   This creates THOUSANDS of lines of output in our daily email.   Logwatch 7.3.6 helps mitigate this.

Logwatch 7.3.6 has a lot of new reports that default to “summary mode”.  You see a single line entry for each notable event, v. a line for each time the event occured.  For instance we see a report more like this for IMAPD..

 [IMAPd] Logout stats:
 ====================
 User | Logouts | Downloaded |  Mbox Size
 --------------------------------------- | ------- | ---------- | ----------
 cpanel@localhost |     287 |          0 |          0
 xyz@cybersprocket.com |       4 |          0 |          0
 ---------------------------------------------------------------------------
 291 |          0 |          0

Versus the older output like this:

--------------------- IMAP Begin ------------------------
 **Unmatched Entries**
LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[32811], protocol=IMAP: 1 Time(s)
LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[32826], protocol=IMAP: 1 Time(s)
LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[32981], protocol=IMAP: 1 Time(s)

LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[32988], protocol=IMAP: 1 Time(s)

LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[33040], protocol=IMAP: 1 Time(s)

LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[33245], protocol=IMAP: 1 Time(s)

LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[33294], protocol=IMAP: 1 Time(s)

LOGIN, user=cpanel@localhost, ip=[::ffff:127.0.0.1], port=[33310], protocol=IMAP: 1 Time(s)
 repeat 280 more times...
 

So as you can imagine, with 10 sections to our logwatch report, the new summary reports make our email a LOT easier to scan for potential problems in our log files.

Upgrading Logwatch

In order to get these cool new features you need to spend 10 minutes, 5 if you’re good with command line Linux, and install the latest version of logwatch. In essence you are downloading a tarzip that is full of new shell and Perl script files.  The install does not compile anything, it simply copies scripts files to the proper directory on your server.

Our example here are all based on the default CentOS 5 paths.

  • Go to a temp install or source directory on your server.
    # cd /usr/local/src
  • Get the source for logwatch
    # wget http://downloads.sourceforge.net/project/logwatch/logwatch-7.3.6.tar.gz?use_mirror=iweb
  • Extract the files
    # tar xvfz logwatch-7.3.6.tar.gz
  • Make the install script executable
    # cd logwatch-7.3.6
    # chmod a+x install_logwatch.sh
  • Run the script & enter the correct paths for logwatch:
    # ./install_logwatch.sh
    ...Logwatch Basedir [/usr/share/logwatch]  : /etc/log.d
    ...Logwatch ConfigDir [/etc/logwatch] : /etc/log.d
    ...temp files [/var/cache/logwatch] : <enter>
    ...perl [/usr/bin/perl] : <enter>
    ...manpage [/usr/share/man] : <enter>

Conclusion

That’s it.  You should now be on the latest version of logwatch.

You can tweak a lot of the settings by editing the files in /etc/log.d/default.conf/services/<service-name>, for example we ask logwatch to only tell us when someones attempt to connect to our server has been dropped more than 10 times by our Fireant scripts (we do this via the iptables service setting).

Hope you find this latest update useful.   We certainly did!

Posted on

IP Based Firewall with cPanel

CPanel/WHM Based Systems

If you are using a web server from a web hosting company, chances are the CPanel/WHM is the system admin interface you use to manage your server.

The current revision of CPanel/WHM (Mar 5th, 2008) appears to rely on the host access file as a method of preventing access to the system. Access to iptables or ipchains rules is not readily apparent, however it is possible that we have overlooked these options.

Blocking An IP Range

The steps below will help you research who is connecting to your box and how to block them from gaining access to your system through software based IP blocking.

Real World Example

This implementation is based on our experiences after turning on the Logwatch utility on our web server. The logwatch report for PAM shows sshd authentication failures. From our most recent report:

--------------------- pam_unix Begin ------------------------
sshd:
  Authentication Failures:
     unknown (210.205.231.78): 45 Time(s)
     root (210.205.231.78): 10 Time(s)
     unknown (202.118.6.126): 9 Time(s)
     ftp (202.118.6.126): 4 Time(s)
     mail (202.118.6.126): 4 Time(s)
     root (c-68-58-191-51.hsd1.sc.comcast.net): 2 Time(s)
     apache (210.205.231.78): 1 Time(s)
     ftp (210.205.231.78): 1 Time(s)
     mysql (210.205.231.78): 1 Time(s)
     named (210.205.231.78): 1 Time(s)
     postgres (210.205.231.78): 1 Time(s)
  Invalid Users:
     Unknown Account: 54 Time(s)
---------------------- pam_unix End -------------------------

The first entry concerns us since there were 45 attempts to access our system that failed. We check the IP range doing a whois lookup (we use DNS Stuff to do our homework) to determine whether or not a general IP block makes sense. We then use CPanel/WHM utilities to shut down access from the offending IP.

Note: This procedure can prevent ANYONE from accessing your server, including yourself, if not done correctly. If you are not confident in your abilities do not even attempt this. Or as the boys like to say “Don’t attempt anything we’re about to do at home. EVER!”

WHM Host Access Control

Enlarge

WHM Host Access Control

  1. Run a DNSStuff whois lookup:
  2. Connect to our CPanel/WHM service via the web connection that our hosting company gave us (http://host.<domain>.com:2086).
  3. Click on the security icon
  4. Click on security center
  5. Click on host access control
  6. In the four entry boxes that are presented, type:
    • daemon : ALL (do not let them connect to ANYTHING on this box, even the web ports)
    • access list: 210.205.231.78/255.255.255.0 (block anyone connecting from 210.205.231.*)
      • Based on our whois lookup we know that all ip addresses under the 210.205.231.* range are from a specific ISP in Korea. While all the users under that range may not be bad guys, we know from experience that the hackers may get a different IP next week as they tend to be assigned their IP address dynamically. We prefer to block a few of the good guys to shut down the one nuisance user. Your beliefs in the goodness of humanity may dictate a different strategy.
    • action: deny (versus allow which would always let them in regardless of other rules)
    • comment: Korea (you can enter whatever you’d like)
  7. Click Save Host Access List on the bottom of the screen

Go back into security center and click Host Access List. Verify your latest entry appears and that the data is correct. If it is entered incorrectly you may block legitimate users from accessing your system.

Turning On Logwatch

Logwatch notifications may not be enabled on your CPanel/WHM system. Logwatch tends to be running in the background but the notifications go to Never-Never Land by default. You will need to look in system notifications and enter an email address to actually see your messages.

Concepts

Software Based IP Blocking

Software based IP blocking is a method for preventing access to your system by using a program running on the target computer (the computer people are trying to hack) that intercepts the connection by hooking into the TCP/IP process flow.

Software based IP blocking will consume CPU resources and memory on the target box. It can also be susceptible to hacking, although this is unusual, because it is nothing more than another program that runs on the server. For these reasons, many people consider a separate hardware firewall appliance as the better solution.

However, many web hosting services do not offer external firewall appliances. Those that do may charge more than you are willing to spend on security. In these cases you can still protect yourself via a software based IP blocking program. The most common options on Linux boxes are to use a software based firewall (ipchains or iptables) or preventing connections via host access directives.

Implementation of these concepts is discussed elsewhere on this page.