Published July 1st, 2010 by admin
Unfortunately, we still need to maintain some old PHP4 apps for customers along with PHP5 apps. This causes a lot of grief for in-house developers who tend to hate to support old PHP4 stuff. On top of it, maintaining PHP5 and PHP4 modules for Apache on same development server is a painful experience to say the least.
If you are in similar boat and need to run PHP5 + PHP4 modules with Apache 2.x on the same box, here are a step-by-step summary of what you can do to get this done:
- Install Apache 2.x source in /usr/local/src/httpd-
- Configure, compile and install apache in /home/apache
- Install PHP5 source on /usr/local/src/php5-
- Configure, compile and install PHP5 module using /home/apache/bin as the apxs path
- Create /usr/local/lib/php5 and copy a php.ini sample from the /usr/local/src/php5 directory to it
- Configure /home/apache/conf/httpd.conf to have
PHPIniDir /usr/local/lib/php5 directive and also configure Apache to either listen to a specific IP address or port
- Start Apache as usual
/home/apache/bin/apachectl start command
- Create a PHP script with phpinfo() call and verify that Apache is loading PHP5 module and also pointing to /usr/local/lib/php5 for the php.ini file
At this point, if you want to install support for some PECL extensions such as memcache, you want to get them done. For example, to install memcache extension for PHP do the following;
- Run
pecl install memcache
- Copy the memcache.so in /usr/local/lib/php5/memcache.so
- Edit the /usr/local/lib/php5/php.ini file to load the memcache.so extension from the newly created directory in the last step
Now repeat the above process for PHP4 as follows:
- Install Apache 2.x source in /usr/local/src/legacy/httpd-
- Configure, compile and install apache in /home/httpd
- Install PHP5 source on /usr/local/src/legacy/php4-
- Configure, compile and install PHP4 module using /home/httpd/bin as the apxs path
- Create /usr/local/lib/php4 and copy a php.ini sample from the /usr/local/src/php4 directory to it
- Configure /home/httpd/conf/httpd.conf to have
PHPIniDir /usr/local/lib/php4 directive and also configure Apache to listen to a different IP address or a different port
- Start Apache as usual using
/home/httpd/bin/apachectl start command
- Create a PHP script with phpinfo() call and verify that Apache is loading PHP4 module and also pointing to /usr/local/lib/php4 for the php.ini file
At this point, if you want to install support for some PECL extensions such as memcache, you want to get them done. For example, to install memcache extension for PHP do the following;
- Run
pecl install memcache
- Copy the memcache.so in /usr/local/lib/php4/memcache.so
- Edit the /usr/local/lib/php4/php.ini file to load the memcache.so extension from the newly created directory in the last step
You should now have both PHP5 and PHP4 support for Apache 2.x on your Linux server.
Enjoy!
Happy 4th of July 2010
Category: Virtual Host | Tags: | Be the First to Comment »
Published March 2nd, 2010 by admin
Google Analtyics, Quantcast etc are nice for marketing types. We needed a quick-and-dirty bandwidth report based on Apache access log. So whipped out the following perl script to do just that.
#!/usr/bin/env perl
use Time::DaysInMonth;
$|++;
my %months = ('Jan' => 1,
'Feb' => 2,
'Mar' => 3,
'Apr' => 4,
'May' => 5,
'Jun' => 6,
'Jul' => 7,
'Aug' => 8,
'Sep' => 9,
'Oct' => 10,
'Nov' => 11,
'Dec' => 12);
my %monByteStats;
my %monRequestStats;
my $cnt;
my $lastMonKey;
my $lastYear;
my $lastMon;
while(my $line = <STDIN>)
{
my (@fields) = split(/\s+/, $line);
my $ip = $fields[0];
my $status = $fields[-2];
my $bytes = $fields[-1];
# seprate date and time from timestamp
my ($date, $time) = split(/:/, $fields[3]);
$date =~ s/\[//;
my ($day, $mon, $year) = split(/\//, $date);
$key = $mon . '-' . $year;
$monRequestStats{$key}++;
$cnt++;
$lastMonKey = $key if ($lastMonKey eq "");
$lastMon = $mon if ($lastMon eq "");
$lastYear = $year if ($lastYear eq "");
if ($lastMonKey ne $key)
{
showProgressReport($months{$lastMon},
$lastMon,
$lastYear,
$monByteStats{$lastMonKey},
$monRequestStats{$lastMonKey});
$lastMon = $mon;
$lastYear = $year;
$lastMonKey = $key;
}
$monByteStats{$key} += $bytes;
}
print "\n";
foreach my $monYear (keys %monByteStats)
{
$totalBytes = $monByteStats{$monYear};
print "$monYear $totalBytes\n";
}
exit;
sub showProgressReport
{
my $monNum = shift;
my $mon = shift;
my $year = shift;
my $totalBytes = shift;
my $requests = shift;
my $totalDays = days_in($year, $monNum);
my $MB = sprintf("%.2f", $totalBytes / (1024*1024));
my $GB = sprintf("%.2f", $MB / 1024);
my $totalBits = $totalBytes * 8;
my $totalKiloBits = $totalBits / 1024;
my $totalMegaBits = $totalKiloBits / 1024;
my $megaBitsPerDay = sprintf("%.2f", $totalMegaBits / $totalDays);
my $megaBitsPerHour = sprintf("%.2f", $megaBitsPerDay / 24);
my $megaBitsPerSec = sprintf("%.2f", $megaBitsPerHour / 3600);
print "$mon $year sent: $GB GB ($MB MB) " .
"requests: $requests bandwidth: $megaBitsPerSec Mbps...\n";
}
Here is a sample report:
Oct 2008 sent: 0.02 GB (15.66 MB) requests: 3887 bandwidth: 0.00 Mbps...
Nov 2008 sent: 17.61 GB (18037.23 MB) requests: 1024067 bandwidth: 0.06 Mbps...
Dec 2008 sent: 153.64 GB (157324.32 MB) requests: 7422782 bandwidth: 0.47 Mbps...
Category: Log Management | Tags: | Be the First to Comment »
Published January 25th, 2010 by admin
Today we found out that one of the client sites had their precious hacker-safe badge dropped because the Web server that they were running was offering SSL v2 support.
So we decided to investigate by running:
$ openssl s_client –ssl2 –connect www.clienthostname.com:443
We were able to connect! This confirmed that we had SSL v2 enabled Apache, which is default when you compile SSL module. So we decided to disable SSL v2 support by changing the following SSL directives in virtual host configuration that answers to port 443:
SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM
Once we restarted Apache, we tested using the above-mentioned command to see if SSL v2 was still enabled or not and this time no connection was available.
Just to see if SSL v3 and TLS v1 are now enabled, we ran:
For SSL v3:
$ openssl s_client –ssl3 –connect www.clienthostname.com:443
For TLS v1:
$ openssl s_client –tls1 –connect www.clienthostname.com:443
Here is the sample output:
SSL handshake has read 5369 bytes and written 327 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
Protocol : SSLv3
Cipher : DHE-RSA-AES256-SHA
Session-ID:
Session-ID-ctx:
Master-Key: E0DECA41A91BF2BD174C0564FDAE447B1D3B254FDCD47948E082B590562329A0BBF61FD24F7985A161887751C0E5C8EE
Key-Arg : None
Krb5 Principal: None
Compression: 1 (zlib compression)
Start Time: 1264476556
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
---
closed
Category: SSL, Security, Virtual Host | Tags: | Be the First to Comment »
Published January 16th, 2010 by admin
We do not run local log analysis program since most customers rely on off-site Google Analytics. So we find ourselves often writing on-the-fly awk scripts to find the top-100 or top-1000 IP visitors from time to time when debugging site performance issues. Who hasn’t found their sites slow thanks to crawlers jamming the site?
So to avoid writing ugly awk script in the middle of the night every so often, here is a Perl script that takes the Apache log file name as input and analyzes to report the top IP visitors. You can even limit how many you want to see by passing a second parameter.
#!/bin/env perl
use Cwd;
use Net::DNS;
use Socket;
my $progressMarker = 100000;
my $res = Net::DNS::Resolver->new;
$| = 1;
my $inputFile = $ARGV[0];
my $limit = $ARGV[1];
my $logFile = getcwd . '/' . $inputFile;
my %stats;
my $topLimit = 0;
if ($limit ne "")
{
$topLimit = $limit;
}
print "Scanning $inputFile ....";
my $progressCnt = 0;
open(LOG, $logFile) || die "Cannot open $logFile $!";
my $totalHits = 0;
while(my $line = )
{
my @fields = split(/\s/, $line);
if ($progressCnt % $progressMarker == 0)
{
print '.';
}
my $ip = $fields[0];
$progressCnt++;
$stats{$ip}++;
$totalHits++;
}
close(LOG);
print "done.\n";
print "Sorting IP by visit count ...";
@ipList = sort { $stats{$b} <=> $stats{$a} } keys %stats;
print "done.\n";
printf("%-10s%-20s%-60s%-10s%-5s\n","RANK",
"IP ADDR",
"HOST",
"VISITS",
"% OF VISITS");
my $rank = 0;
foreach my $thisIP (@ipList)
{
my $cnt = $stats{$thisIP};
my $host = "n/a";
my $reverseIP = join('.', reverse split(/\./, $IP)).".in-addr.arpa";
my $query = $res->query("$reverseIP", "PTR");
if ($query)
{
foreach my $rr ($query->answer)
{
next unless $rr->type eq "PTR";
$host = $rr->rdatastr;
}
}
else
{
$err = $res->errorstring;
if ($err eq "NOERROR")
{
my $iaddr = inet_aton($thisIP);
$host = gethostbyaddr($iaddr, AF_INET);
}
if ($host eq "")
{
$host = "unknown (dig -x $thisIP)";
}
}
$rank++;
printf("%-10d%-20s%-60s%-10d%-4.2f%\n",$rank,
$thisIP,
$host,
$cnt,
($cnt * 100/$totalHits));
if ($rank >= $topLimit)
{
exit;
}
}
Install this Perl script in /usr/bin as apache_top_ip and chmod it to allow appropriate users to be able to run it as:
$ apache_top_requests
Here is an example call to get the top 100 IP addresses with possibly resolved host names from a log file:
$ apache_top_requests /logs/apache-2010-jan.log 100
Enjoy!
Category: Log Management, Security | Tags: | Be the First to Comment »
Published December 4th, 2009 by admin
We needed a prototype filter that allows us to filter Web pages generated by PHP scripts to be post processed using an Apache output filter. In addition, we needed to write the filter in PHP as well. The filter will simply take the HTML (text/html) output generated by a third-party PHP application and do some post processing before sending it out to the browser.
The flow looks like:
Request
|
+--> Apache
|
+---> 3rd Party PHP App
|
+---> Output in text/html
|
+---> Our Apache Output Filter in PHP
|
+---> Response in text/html to Web browser
Now, since this is a prototype, we did not have issue with performance so decided to use Apache’s external filter module by configuring and compiling Apache source using –enable-ext-filter option.
Then configured our target virtual host to have:
ExtFilterDefine magicfilter mode=output cmd=/path/to/filter.php intype=text/html outtype=text/html
<Location />
SetOutputFilter magicfilter
</Location>
Then we wrote a basic filter script called filter.php as follows:
#!/usr/local/bin/php
<?php
$htmlContents = file_get_contents('php://stdin');
// Apply changes to $htmlContents
// code not shown here
echo $htmlContents;
?>
Once the filter script was made executable by the Apache Web server, and Apache was restarted, it ran and did the charm as expected.
Category: Virtual Host | Tags: | Be the First to Comment »
Published December 3rd, 2009 by admin
Just tried to compile Apache on a system with APR utility version 1.0 and got errors. To remedy, just told configure to use --with-included-apr which makes it use the bundled 1.2 APR libraries. Worked like a charm.
Category: Uncategorized | Tags: | Be the First to Comment »
Published July 25th, 2009 by admin
We planned to upgrade an existing 64-bit CentOS 5 server this weekend. The upgrade caused a bit of a headache that is worth documenting for future reference.
Getting CentOS 5 Upgraded to CentOS 5.3
This went like a breeze as all we did is run: nohup yum –y upgrade & and in the background the CentOS upgrade continued for about 30 minute – just perfect for a coffee in front and a bit of channel surfing. Once the upgrade was done, the fun began.
Getting Apache 2.2.11 Compiled
Typically Apache install on a 32-bit system is very peaceful and simple. However, when we tried to compile Apache, it keeps failing to find the right 64-bit library files. So we included the –with-lib=/lib64 in the configuration command and also made sure that /etc/ld.so.conf has a line: /lib64 in it. Even we ran: ldconf –vp to make ld’s library cache file update.
However, we were getting the same Berkeley DB missing error that pointed to a library in the /usr/lib which stores 32-bit version. CentOS yum updates install both 32-bit and 64-bit versions of the same library. But even with clear instructions to Apache configuration command to load libraries from /lib64 path did not do much to help.
Finally after digging a lot on the net, we found that there is a Apache run-time developer’s package that got installed that is the culprit. Running: yum list apr-devel verified that we have the culprit. So just running: yum remove apr-devel to get rid of the package and running: configure && make && make install for Apache did the magic tric.
Category: Uncategorized | Tags: | Be the First to Comment »
Published July 11th, 2009 by admin
I wanted to to have a PHP script fire every time someone went to a specific URL which was not a real directory so I came up with a simple Apache configuration to deal with this scenario. Here is an example of such a configuration:
<Location /start>
SetHandler start-app
Action start-app /start.php virtual
</Location>
When a user visits http://server/start/here or http://server/start/there etc. the /start.php script found on the document root runs. In the script, I can then determine what file is requested by taking a peet at the REQUEST_URI and deal with the file request accordingly.
Just thought I share this little tip for you!
Enjoy.
Kabir
Category: Virtual Host | Tags: | Be the First to Comment »
Published December 14th, 2008 by kabir
Problem Statement
When you are developing Web applications, Apache logs — access log and/or error log — can be really useful tool. However, by default Apache logs a lot of stuff that gets in the way of debugging if you are focused on solving a specific problem with your Web app. In this article, we will show you how you can write a very simple PHP script to customize what is logged or not.
Creating a PHP Script for Processing Apache Log in Real Time
Instead of creating a PHP based Apache log parser, which would be not in real-time, we will create a simple PHP script as shown below to process Apache log entries in real-time as they are created by Apache log module. Look at the following PHP script in Listing 1.
Listing 1: simple_apache_logger.php
#!/bin/env php
<?
$logDir = '/var/data/logs';
$logFile = $logDir . '/access.log';
$fp = fopen($logFile,"a+");
$stdin = fopen("php://stdin", "r");
// Use unbuffered output
ob_implicit_flush (true);
while ($line = fgets($stdin))
{
fwrite($fp, $line);
}
fclose($fd);
fclose($stdin);
?>
To run this script, modify your CustomLog entry to be:
CustomLog "|/path/to/simple_apache_logger.php" common
Make sure that the script is executable by running chmod 750 simple_apache_logger.php.
What this script does is as follows:
- Opens a log file called $logFile in append mode in $logDir using $fp file pointer
- Open the STDIN (standard input) as a file called $stdin
- Tells PHP to flush output every time a file I/O is done
- In a while loop, reads a line of data from $stdin into a variable called $line
- The $line is then appended to the log file
When run, this script is loaded once and keeps on running as long as Apache runs. So there is no load cost per log entry. It runs and appends the same log data given by Apache to a file. This means nothing interesting is being done in this version of the script as it is simply writing a log file, which would be identical to the original Apache log file. So to makes this interesting, lets update Listing 1 as shown in Listing 2.
Listing 2: modified while() loop for simple_apache_logger.php
while ($line = fgets($stdin))
{
// Ignore all log requests for common image, cascading stylesheets, JavaScripts, and flash video
if (preg_match("/(\w+)\.(gif|png|jpg|css|js|swf)/", $line))
{
continue;
}
If you replace the original while() loop that simply wrote the Apache log entry in a file to the above-mentioned while() loop that ignores the common image, cascading style sheets, and JavaScript requests from being logged, you end up with a clean log of requests for pages instead of recording all the external components (images, JavaScripts, CSS) that make up an HTML page. The reduction in log entries makes it easier to deal with debugging GET parameters or other SEO related matter much easier.
Creating a PHP Script for Rotating Apache Log
Unfortunately, you cannot pipe multiple programs with CustomLog to do something like:
CustomLog "|/usr/local/sbin/cronolog /logs/%Y/%m/%d/access.log|/path/to/simple_apache_logger.php" common
So when using a PHP logging tool, you cannot use CronoLog. In such a case, you might need to invent your own log rotation schema. For example, Listing 3 shows an updated version of simple_apache_logger.php that does exactly that.
Listing 1: simple_apache_logger.php
#!/bin/env php
<?
$logDir = '/var/data/logs/ekblogs';
$logFile = $logDir . '/access.'. date("d-m-Y") . '.log';
$fp = fopen($logFile,"a+");
$stdin = fopen("php://stdin", "r");
// Use unbuffered output
ob_implicit_flush (true);
$lastLogDate = date('Ymd');
while ($line = fgets($stdin))
{
// Ignore images, javascripts, css, ico and flash file requests
if (preg_match("/(\w+)\.(gif|png|jpg|css|js|swf|ico)/", $line))
{
continue;
}
// Following section is for rotating log when day change is detected
// This will *only rotate* if requests are coming in daily (which is expected)
$today = date('Ymd');
// If today is different than last log date, time to write a new log file
if ($today > $lastLogDate )
{
$lastLogDate = $today;
fclose($fp);
$logFile = $logDir . '/access.'. date("d-m-Y") . '.log';
$fp = fopen($logFile, "a+");
}
fwrite($fp, $line);
}
fclose($fd);
fclose($stdin);
?>
Every time Apache injects a log entry into the STDIN of this script, it checks if current date is same as the last date it had. If the current date has changed, it then creates a new log file and sets the last date to current date. This allows it to rotate the logs by day.
However, this works only when you have reasonable expectations that your site will get hit every day. If your site does not get hit every day, you will have log entries going into previous log file as the script only gets data to process when there is a new request. But this is not too bad as we only recommend this kind of PHP script based logging in development environments.
Category: Log Management | Tags: | Be the First to Comment »
Published December 13th, 2008 by kabir
Problem Statement
Often, we need to quickly analyze the Apache log files for certain sites without running extensive log analysis program that take a long time to run or runs on a schedule. For quick and dirty probing of Apache logs, you are better of with simple command-line tools. Here we will show you how to perform a few quick and dirty log analysis using standard Linux commands and a scripting language called Awk.
Finding unique IP addresses for a given day
Say you want to find out which unique IP addresses visited your site for a given day, you can run the following command from your shell prompt:
$ grep '[date string]' /path/to/access.log | awk '{print $1}' | sort | uniq
For example, to find out the list of unique IP addresses that have visited the ApacheHacker.com blog yesterday (12/Dec/2008), we can run:
grep '12/Dec/2008' /logs/apachehacker/access.log | awk '{print $1}' | sort | uniq
Finding which IP address visited how many times for a given day
To find out which IP address visited your site how many times a day, run:
$ grep '[date string]' /path/to/access.log | \
awk '{cnt[$1]++;} END{for (ip in cnt){printf("%-15s visited: %04d time(s).\n", ip, cnt[ip])}}'
For example, to find out which IP address visited the ApacheHacker.com blog on Dec 12, 2008, we can run:
$ grep '12/Dec/2008' /logs/apachehacker/access.log | \
awk '{cnt[$1]++;} END{for (ip in cnt){printf("%-15s visited: %04d time(s).\n", ip, cnt[ip])}}'
Here is a sample output:
74.6.8.116 visited: 0001 time(s).
74.6.18.246 visited: 0001 time(s).
93.126.3.33 visited: 0016 time(s).
80.48.192.249 visited: 0044 time(s).
69.62.207.163 visited: 0058 time(s).
89.45.49.247 visited: 0022 time(s).
82.159.52.211 visited: 0015 time(s).
71.226.202.64 visited: 0017 time(s).
203.83.248.74 visited: 0002 time(s).
206.196.125.113 visited: 0001 time(s).
76.188.138.185 visited: 0016 time(s).
203.112.77.18 visited: 0006 time(s).
142.179.135.57 visited: 0016 time(s).
91.121.201.145 visited: 0016 time(s).
74.6.18.215 visited: 0002 time(s).
66.249.73.21 visited: 0001 time(s).
66.150.96.121 visited: 0027 time(s).
75.147.236.233 visited: 0002 time(s).
75.119.230.207 visited: 0020 time(s).
203.129.155.4 visited: 0022 time(s).
142.166.3.122 visited: 0002 time(s).
64.1.215.163 visited: 0003 time(s).
Category: Access Control | Tags: | Be the First to Comment »
