Disabling SSL v2 in Apache/SSL

Today we found out that one of the client sites had their precious hacker-safe badge dropped because the Web server that they were running was offering SSL v2 support.

So we decided to investigate by running:

$ openssl s_client –ssl2 –connect www.clienthostname.com:443

We were able to connect! This confirmed that we had SSL v2 enabled Apache, which is default when you compile SSL module. So we decided to disable SSL v2 support by changing the following SSL directives in virtual host configuration that answers to port 443:

SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM

Once we restarted Apache, we tested using the above-mentioned command to see if SSL v2 was still enabled or not and this time no connection was available.

Just to see if SSL v3 and TLS v1 are now enabled, we ran:

For SSL v3:

$ openssl s_client –ssl3 –connect www.clienthostname.com:443

For TLS v1:

$ openssl s_client –tls1 –connect www.clienthostname.com:443

Here is the sample output:

SSL handshake has read 5369 bytes and written 327 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol  : SSLv3
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID:
    Session-ID-ctx:
    Master-Key: E0DECA41A91BF2BD174C0564FDAE447B1D3B254FDCD47948E082B590562329A0BBF61FD24F7985A161887751C0E5C8EE
    Key-Arg   : None
    Krb5 Principal: None
   Compression: 1 (zlib compression)
    Start Time: 1264476556
    Timeout   : 7200 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
closed

Category: SSL, Security, Virtual Host | Tags: | Be the First to Comment »

Quick and Dirty Apache Log Analyzer to Find Top-100 IP Visits

We do not run local log analysis program since most customers rely on off-site Google Analytics. So we find ourselves often writing on-the-fly awk scripts to find the top-100 or top-1000 IP visitors from time to time when debugging site performance issues. Who hasn’t found their sites slow thanks to crawlers jamming the site?

So to avoid writing ugly awk script in the middle of the night every so often, here is a Perl script that takes the Apache log file name as input and analyzes to report the top IP visitors. You can even limit how many you want to see by passing a second parameter.

#!/bin/env perl

use Cwd;
use Net::DNS;
use Socket;

my $progressMarker = 100000;
my $res = Net::DNS::Resolver->new;

$| = 1;

my $inputFile = $ARGV[0];
my $limit     = $ARGV[1];

my $logFile = getcwd . '/' . $inputFile;

my %stats;
my $topLimit = 0;

if ($limit ne "")
{
    $topLimit = $limit;
}

print "Scanning $inputFile ....";

my $progressCnt = 0;
open(LOG, $logFile) || die "Cannot open $logFile $!";

my $totalHits = 0;
while(my $line = )
{
   my @fields = split(/\s/, $line);

   if ($progressCnt % $progressMarker == 0)
   {
       print '.';
   }

   my $ip = $fields[0];
   $progressCnt++;
   $stats{$ip}++;
   $totalHits++;
}

close(LOG);

print "done.\n";

print "Sorting IP by visit count ...";
@ipList = sort { $stats{$b} <=> $stats{$a} } keys %stats;
print "done.\n";

printf("%-10s%-20s%-60s%-10s%-5s\n","RANK",
                                                       "IP ADDR",
                                                       "HOST",
                                                       "VISITS",
							"% OF VISITS");

my $rank = 0;

foreach my $thisIP (@ipList)
{
   my $cnt       = $stats{$thisIP};
   my $host      = "n/a";
   my $reverseIP = join('.', reverse split(/\./, $IP)).".in-addr.arpa";
   my $query     = $res->query("$reverseIP", "PTR");

   if ($query)
   {
      foreach my $rr ($query->answer)
      {
         next unless $rr->type eq "PTR";
         $host = $rr->rdatastr;
      }
   }
   else
   {
      $err  = $res->errorstring;

      if ($err eq "NOERROR")
      {
          my $iaddr = inet_aton($thisIP);
          $host  = gethostbyaddr($iaddr, AF_INET);
      }
      if ($host eq "")
      {
          $host = "unknown (dig -x $thisIP)";
      }
   }

   $rank++;

   printf("%-10d%-20s%-60s%-10d%-4.2f%\n",$rank,
                                                               $thisIP,
                                                               $host,
                                                               $cnt,
                                                               ($cnt * 100/$totalHits));

   if ($rank >= $topLimit)
   {
       exit;
   }
}

Install this Perl script in /usr/bin as apache_top_ip and chmod it to allow appropriate users to be able to run it as:


$ apache_top_requests


Here is an example call to get the top 100 IP addresses with possibly resolved host names from a log file:


$ apache_top_requests /logs/apache-2010-jan.log 100


Enjoy!

Category: Log Management, Security | Tags: | Be the First to Comment »

Output Filtering Using PHP

We needed a prototype filter that allows us to filter Web pages generated by PHP scripts to be post processed using an Apache output filter.  In addition, we needed to write the filter in PHP as well. The filter will simply take the HTML (text/html) output generated by a third-party PHP application and do some post processing before sending it out to the browser.

The flow looks like:

Request
|
+--> Apache
      |
      +---> 3rd Party PHP App
               |
               +---> Output in text/html
                        |
                        +---> Our Apache Output Filter in PHP
                                 |
                                 +---> Response in text/html to Web browser

Now, since this is a prototype, we did not have issue with performance so decided to use Apache’s external filter module by configuring and compiling Apache source using –enable-ext-filter option.

Then configured our target virtual host to have:

   ExtFilterDefine magicfilter mode=output cmd=/path/to/filter.php intype=text/html outtype=text/html
    <Location />
       SetOutputFilter magicfilter
    </Location>

Then we wrote a basic filter script called filter.php as follows:

#!/usr/local/bin/php
<?php

   $htmlContents = file_get_contents('php://stdin');

   // Apply changes to $htmlContents
   // code not shown here

   echo  $htmlContents;
?>

Once the filter script was made executable by the Apache Web server, and Apache was restarted, it ran and did the charm as expected.

Category: Virtual Host | Tags: | Be the First to Comment »

Compiling Apache with Old Version of APR Utils…

Just tried to compile Apache on a system with APR utility version 1.0 and got errors. To remedy, just told configure to use --with-included-apr which makes it use the bundled 1.2 APR libraries. Worked like a charm.

Category: Uncategorized | Tags: | Be the First to Comment »

Another 64-bit Headache Installing Apache on CentOS 5.3

We planned to upgrade an existing 64-bit CentOS 5 server this weekend. The upgrade caused a bit of a headache that is worth documenting for future reference.

Getting CentOS 5 Upgraded to CentOS 5.3

This went like a breeze as all we did is run: nohup yum –y upgrade & and in the background the CentOS upgrade continued for about 30 minute – just perfect for a coffee in front and a bit of channel surfing. Once the upgrade was done, the fun began.

Getting Apache 2.2.11 Compiled

Typically Apache install on a 32-bit system is very peaceful and simple. However, when we tried to compile Apache, it keeps failing to find the right 64-bit library files. So we included the –with-lib=/lib64 in the configuration command and also made sure that /etc/ld.so.conf has a line: /lib64 in it. Even we ran: ldconf –vp to make ld’s library cache file update.

However, we were getting the same Berkeley DB missing error that pointed to a library in the /usr/lib which stores 32-bit version. CentOS yum updates install both 32-bit and 64-bit versions of the same library. But even with clear instructions to Apache configuration command to load libraries from /lib64 path did not do much to help.

Finally after digging a lot on the net, we found that there is a Apache run-time developer’s package that got installed that is the culprit.  Running:  yum list apr-devel verified that we have the culprit. So just running: yum remove apr-devel to get rid of the package and running: configure && make && make install for Apache did the magic tric.

Category: Uncategorized | Tags: | Be the First to Comment »

Setting up a PHP Filter for a Directory Alias

I wanted to to have a PHP script fire every time someone went to a specific URL which was not a real directory so I came up with a simple Apache configuration to deal with this scenario. Here is an example of such a configuration:

  <Location /start>
    SetHandler start-app
    Action start-app /start.php virtual
  </Location>

When a user visits http://server/start/here  or http://server/start/there etc. the /start.php script found on the document root runs.  In the script, I can then determine what file is requested by taking a peet at the REQUEST_URI and deal with the file request accordingly.

Just thought I share this little tip for you!

Enjoy.

Kabir

Category: Virtual Host | Tags: | Be the First to Comment »

Writing a Custom Apache Log File Using a PHP Script in Real Time

Problem Statement

When you are developing Web applications, Apache logs — access log and/or error log — can be really useful tool. However, by default Apache logs a lot of stuff that gets in the way of debugging if you are focused on solving a specific problem with your Web app. In this article, we will show you how you can write a very simple PHP script to customize what is logged or not.

Creating a PHP Script for Processing Apache Log in Real Time

Instead of creating a PHP based Apache log parser, which would be not in real-time, we will create a simple PHP script as shown below to process Apache log entries in real-time as they are created by Apache log module. Look at the following PHP script in Listing 1.

Listing 1: simple_apache_logger.php

#!/bin/env php
<?
$logDir  = '/var/data/logs';
$logFile = $logDir . '/access.log';
$fp      = fopen($logFile,"a+");
$stdin   = fopen("php://stdin", "r");

// Use unbuffered output
ob_implicit_flush (true);

while ($line = fgets($stdin))
{
   fwrite($fp, $line);
}

fclose($fd);
fclose($stdin);
?>

To run this script, modify your CustomLog entry to be:

CustomLog "|/path/to/simple_apache_logger.php" common

Make sure that the script is executable by running chmod 750 simple_apache_logger.php.
What this script does is as follows:

1. Opens a log file called $logFile in append mode in $logDir using $fp file pointer
2. Open the STDIN (standard input) as a file called $stdin
3. Tells PHP to flush output every time a file I/O is done
4. In a while loop, reads a line of data from $stdin into a variable called $line
5. The $line is then appended to the log file

When run, this script is loaded once and keeps on running as long as Apache runs. So there is no load cost per log entry. It runs and appends the same log data given by Apache to a file. This means nothing interesting is being done in this version of the script as it is simply writing a log file, which would be identical to the original Apache log file. So to makes this interesting, lets update Listing 1 as shown in Listing 2.

Listing 2: modified while() loop for simple_apache_logger.php

while ($line = fgets($stdin))
{
   // Ignore all log requests for common image, cascading stylesheets, JavaScripts, and flash video
   if (preg_match("/(\w+)\.(gif|png|jpg|css|js|swf)/", $line))
   {
       continue;
}

If you replace the original while() loop that simply wrote the Apache log entry in a file to the above-mentioned while() loop that ignores the common image, cascading style sheets, and JavaScript requests from being logged, you end up with a clean log of requests for pages instead of recording all the external components (images, JavaScripts, CSS) that make up an HTML page. The reduction in log entries makes it easier to deal with debugging GET parameters or other SEO related matter much easier.

Creating a PHP Script for Rotating Apache Log

Unfortunately, you cannot pipe multiple programs with CustomLog to do something like:

CustomLog "|/usr/local/sbin/cronolog /logs/%Y/%m/%d/access.log|/path/to/simple_apache_logger.php" common

So when using a PHP logging tool, you cannot use CronoLog. In such a case, you might need to invent your own log rotation schema. For example, Listing 3 shows an updated version of simple_apache_logger.php that does exactly that.
Listing 1: simple_apache_logger.php

#!/bin/env php
<?

$logDir  = '/var/data/logs/ekblogs';
$logFile = $logDir . '/access.'. date("d-m-Y") . '.log';
$fp      = fopen($logFile,"a+");
$stdin   = fopen("php://stdin", "r");

// Use unbuffered output
ob_implicit_flush (true);

$lastLogDate = date('Ymd');

while ($line = fgets($stdin))
{
   // Ignore images, javascripts, css, ico and flash file requests
   if (preg_match("/(\w+)\.(gif|png|jpg|css|js|swf|ico)/", $line))
   {
       continue;
   }

   // Following section is for rotating log when day change is detected
   // This will *only rotate* if requests are coming in daily (which is expected)

   $today = date('Ymd');

   // If today is different than last log date, time to write a new log file
   if ($today > $lastLogDate )
   {
       $lastLogDate = $today;
       fclose($fp);
       $logFile = $logDir . '/access.'. date("d-m-Y") . '.log';
       $fp      = fopen($logFile, "a+");
   }

   fwrite($fp, $line);
}

fclose($fd);
fclose($stdin);
?>

Every time Apache injects a log entry into the STDIN of this script, it checks if current date is same as the last date it had. If the current date has changed, it then creates a new log file and sets the last date to current date. This allows it to rotate the logs by day.

However, this works only when you have reasonable expectations that your site will get hit every day. If your site does not get hit every day, you will have log entries going into previous log file as the script only gets data to process when there is a new request. But this is not too bad as we only recommend this kind of PHP script based logging in development environments.

Category: Log Management | Tags: | Be the First to Comment »

Using Awk to do Quick and Dirty Analysis of Apache Logs

Problem Statement

Often, we need to quickly analyze the Apache log files for certain sites without running extensive log analysis program that take a long time to run or runs on a schedule. For quick and dirty probing of Apache logs, you are better of with simple command-line tools. Here we will show you how to perform a few quick and dirty log analysis using standard Linux commands and a scripting language called Awk.

Finding unique IP addresses for a given day

Say you want to find out which unique IP addresses visited your site for a given day, you can run the following command from your shell prompt:

$ grep '[date string]' /path/to/access.log | awk '{print $1}' | sort | uniq

For example, to find out the list of unique IP addresses that have visited the ApacheHacker.com blog yesterday (12/Dec/2008), we can run:

grep '12/Dec/2008' /logs/apachehacker/access.log | awk '{print $1}' | sort | uniq

Finding which IP address visited how many times for a given day

To find out which IP address visited your site how many times a day, run:

$ grep '[date string]' /path/to/access.log |  \
   awk '{cnt[$1]++;} END{for (ip in cnt){printf("%-15s visited: %04d time(s).\n", ip, cnt[ip])}}'

For example, to find out which IP address visited the ApacheHacker.com blog on Dec 12, 2008, we can run:

$ grep '12/Dec/2008' /logs/apachehacker/access.log | \
  awk '{cnt[$1]++;} END{for (ip in cnt){printf("%-15s visited: %04d time(s).\n", ip, cnt[ip])}}'

Here is a sample output:

74.6.8.116      visited: 0001 time(s).
74.6.18.246     visited: 0001 time(s).
93.126.3.33     visited: 0016 time(s).
80.48.192.249   visited: 0044 time(s).
69.62.207.163   visited: 0058 time(s).
89.45.49.247    visited: 0022 time(s).
82.159.52.211   visited: 0015 time(s).
71.226.202.64   visited: 0017 time(s).
203.83.248.74   visited: 0002 time(s).
206.196.125.113 visited: 0001 time(s).
76.188.138.185  visited: 0016 time(s).
203.112.77.18   visited: 0006 time(s).
142.179.135.57  visited: 0016 time(s).
91.121.201.145  visited: 0016 time(s).
74.6.18.215     visited: 0002 time(s).
66.249.73.21    visited: 0001 time(s).
66.150.96.121   visited: 0027 time(s).
75.147.236.233  visited: 0002 time(s).
75.119.230.207  visited: 0020 time(s).
203.129.155.4   visited: 0022 time(s).
142.166.3.122   visited: 0002 time(s).
64.1.215.163    visited: 0003 time(s).

Category: Access Control | Tags: | Be the First to Comment »

Disabling Weak SSL v2 Support in Apache Server

Problem Statement

By default, Apache 2.x with SSL enabled uses SSL v2, which was introduced by  Netscape Communications Corporation with the launch of Netscape Navigator 1.0 in 1994 and it contains several well-known weaknesses. For example, SSLv2 doesn’t provide any protection against man-in-the-middle attacks during the handshake, and uses the same cryptographic keys for message authentication and for encryption. If you use any third-party scanner service such as McAfee, ScanAlert, etc. you will get a high-level vulnerability flag for SSL v2. Here we will show you how to upgrade Apache’s SSL support.

Where is SSL Support Going for Popular Web browsers

By default, Internet Explorer 7 (IE7) disables SSLv2 support and enables the stronger TLSv1 instead.
IE7 will only negotiate HTTPS connections using SSLv3 or TLSv1. Mozilla Firefox is expected to drop support for SSLv2 in its future versions.  Since nearly all Web browsers now support SSLv3, disabling support for the weaker SSL method should have minimal impact. The following browsers support SSLv3:

• Internet Explorer 5.5 or higher (PC)
• Internet Explorer 5.0 or higher (Mac)
• Netscape 2.0 (Domestic) or higher (PC/Mac)
• Firefox 0.8 or higher (PC/Mac/Linux)
• Mozilla 1.7 or higher (PC/Mac/Linux)
• Camino 0.8 or higher (Mac)
• Safari 1.0 or higher (Mac)
• Opera 1.7 or higher (PC/Mac)
• Omniweb 3.0 or higher (Mac)
• Konqueror 2.0 or higher (Linux)

According to https://www.pcisecuritystandards.org/pdfs/pcissc_assessors_nl_2008-11.pdf, an Assessor’s update report, “…it is imperative that an ASV identify the use of SSL 2.0 to transmit cardholder data as a failure.”

Updating Apache Configuration for SSL

For Apache Server, you need to modify the SSLCipherSuite directive in the httpd.conf or ssl.conf file. Here is the configuration you need to add or edit in these files:

SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP

Once added/edited, restart Apache and confirm that your SSL-enabled site is still working as expected.

Category: SSL, Security | Tags: | Be the First to Comment »

Running Web Sites Under Multiple User Accounts with mod_itk

Problem Statement

When lots of people are working on the same Apache Web server running multiple virtual hosts, creating an effective and secure file/dir permission schema is difficult using Linux’s simplistic user/group concepts. Here we will show you how you can run Apache using different Linux user accounts so that each virtual host runs using its owner’s file/dir permissions. This effectively makes each of the virtual hosts more secure as files accessible (r+w) for one virtual host is not accessible from another.

Step 1: Installing mod_itk source

The mod_itk module is available as a source patch for Apache Web server source distribution. So if you are running Apache from a RPM distribution, you cannot use it. It is for those of us who love to compile Apache from source distribution. We will assume you have compiled and installed Apache from a source distribution and the source code is kept at /usr/local/src/httpd-[version]. Follow the steps below:

1. Download the mod_itk patch file from http://mpm-itk.sesse.net/.
2. Change directory to your Apache source distribution and run patch -p1 < /path/to/[downloaded patch file] and run autoconf
3. Edit your config.nice and add: "--with-mpm=itk" \ before the last line. Here is a sample config.nice:

   #! /bin/sh
   #
   # Created by configure
   
   "./configure" \
   "--prefix=/home/apache" \
   "--enable-so" \
   "--with-ssl=/usr" \
   "--enable-ssl" \
   "--enable-deflate" \
   "--disable-cgi" \
   "--enable-rewrite" \
   "--disable-userdir" \
   "--with-mpm=itk" \
   "$@"
   

4. Run compile and install Apache as follows: ./configure && make && make install

Step 2: Configuring your virtual host using a specific user account

Now create a new linux user and group (or you can use an existing one too) for your virtual host that you want to run using a specific user account. Then follow the steps below:

1. Change file/directory ownership of your virtual host’s document root for the chosen Linux user
2. Edit your virtual host configuration file and add the following lines:

   
   <IfModule mpm_itk_module>
       AssignUserId [username] [groupname]
   </IfModule>
   

3. Now restart Apache and access your Web site via a Web browser
4. On the server’s command-line, run: ps auxww | grep httpd and notice that one or more processes are run using the chosen username.

Category: Access Control, Security, Virtual Host | Tags: | 1 Comment »