WEB SOFTWARES REVIEW

Apache Self Linking error

2008-03-11T13:47:00.000-07:00

===========
[11/Mar/2008:16:05:45 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8b mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 (internal dummy connection)"
=========

Fix : Update MaxSpareServers and MinSpareServers

Innodb error: Unable to restart Mysql

2008-01-20T19:55:00.000-08:00

Forcing InnoDB Recovery

If there is database page corruption, you may want to dump your tables from the database with SELECT INTO OUTFILE. Usually, most of the data obtained in this way is intact. Even so, the corruption may cause SELECT * FROM tbl_name statements or InnoDB background operations to crash or assert, or even to cause InnoDB roll-forward recovery to crash. However, you can force the InnoDB storage engine to start up while preventing background operations from running, so that you are able to dump your tables. For example, you can add the following line to the [mysqld] section of your option file before restarting the server :

vi /etc/my.cnf
[mysqld]
innodb_force_recovery = 4

The allowable non-zero values for innodb_force_recovery follow. A larger number includes all precautions of smaller numbers. If you are able to dump your tables with an option value of at most 4, then you are relatively safe that only some data on corrupt individual pages is lost. A value of 6 is more drastic because database pages are left in an obsolete state, which in turn may introduce more corruption into B-trees and other database structures.

* 1 (SRV_FORCE_IGNORE_CORRUPT)

Let the server run even if it detects a corrupt page. Try to make SELECT * FROM tbl_name jump over corrupt index records and pages, which helps in dumping tables.
* 2 (SRV_FORCE_NO_BACKGROUND)

Prevent the main thread from running. If a crash would occur during the purge operation, this recovery value prevents it.
* 3 (SRV_FORCE_NO_TRX_UNDO)

Do not run transaction rollbacks after recovery.
* 4 (SRV_FORCE_NO_IBUF_MERGE)

Prevent also insert buffer merge operations. If they would cause a crash, do not do them. Do not calculate table statistics.
* 5 (SRV_FORCE_NO_UNDO_LOG_SCAN)

Do not look at undo logs when starting the database: InnoDB treats even incomplete transactions as committed.
* 6 (SRV_FORCE_NO_LOG_REDO)

Do not do the log roll-forward in connection with recovery.

You can SELECT from tables to dump them, or DROP or CREATE tables even if forced recovery is used. If you know that a given table is causing a crash on rollback, you can drop it. You can also use this to stop a runaway rollback caused by a failing mass import or ALTER TABLE. You can kill the mysqld process and set innodb_force_recovery to 3 to bring the database up without the rollback, then DROP the table that is causing the runaway rollback.

The database must not otherwise be used with any non-zero value of innodb_force_recovery. As a safety measure, InnoDB prevents users from performing INSERT, UPDATE, or DELETE operations when innodb_force_recovery is greater than 0.

Installing Domain Keys on Cpanel

2008-01-20T14:45:00.000-08:00

Delivering mail to yahoo and google is so hard those days. Almost every server has had this problem even if it is not used to relay spam.
Installing DomainKeys can help your server deliver “clean” emails directly to your user’s inbox.
What are DomainKeys?
Well you will find more information on this topic at: http://antispam.yahoo.com/domainkeys

Installation is simple and it’s done on a domain basis.

How to install DomainKeys on a specific domain.

1. First check that you are running the latest version on RELEASE or CURRENT of cPanel 11.
2. Run the script

/usr/local/cpanel/bin/domain_keys_installer username

Where username is the cPanel user.

If you get an error similar to “Domain keys are not installed on this machine.” you either are not running the latest release or current version of cPanel or you have not converted yet to maildir. Maildir conversion is required before you install DomainKeys.
You will find an article about converting to maildir on this site !

Ok, we just installed DomainKeys for a domain, but how about if we want to install it for all the domains (users)?
Well I found the solution just a few days ago on a public forum. Someone wrote a nice bash script that will parse all the cpanel users and then run the installation for each of them.

for i in `ls /var/cpanel/users` ;do /usr/local/cpanel/bin/domain_keys_installer $i ;done

Ok, but what about if we want that every new created account to have DomainKeys installed. Well this is a bit harder to do.
I recommend editing /scripts/postwwwacct and adding:

my %OPTS = @ARGV;
my $user = $OPTS{’user’};
/usr/local/cpanel/bin/domain_keys_installer $user

Courtesy: http://www.cpanelconfig.com/cpanel-administration/installing-domainkeys-on-a-cpanel-server/

Common Kernel Panic Errors

2007-12-19T05:03:00.000-08:00

Unable to mount root fs and solution The error is common as soon as the compile happens on a new partition and the most common error will be when the grub/lilo configuration is manually edited by the user, it is preferred to keep the settings to the default. If you drive into error then login to the machine using single user mode and have the grub configuration edited such that the drive letter is corresponded, normally adding /dev/hda0 instead of /dev/hda on the LABEL keyword help.

2. No Init found

1. Boot computer. As soon as Grub splash screen comes up, hit up or down arrow key to stop the clock.
2. Press "c" to get the Grub command line like this: grub>
3. Type:
root (
You should get a list of valid drives. If the result fails then assume that the grub does know the details of the root partition.
4. Press the next character corresponding to the drive OR partition where your /boot directory is and then press
5. Repeat step 4 until you have completed the line so it looks like
root (hdx,y)
where x and y are correct drive (x) and partition (y) where your /boot directory is. You should know already that Grub counts drives and partitions starting with zero for the first primary partition and 4 for the first extended partition.
6. If you do not have a separate /boot partition, type:
kernel /boot/vmlin
If you do have a separate /boot partition, do not enter the /boot and instead, just
kernel /vmlin
In either case, when you hit , Grub will complete the filename, or list possible files ONLY if you have correctly entered all other information. If Grub says something like "no files found", go back and correct your error.
7. Repeat step 6 until you have a complete filename but do not press enter yet.
8. Following the kernel filename, on the same line, add ro so the filesystems will be initially mounted read only so that integrity checks may be made without adding corruption, and add information telling Grub where your top level root partition is: / so that the line looks something like this:
kernel /vmlinuz-2.4.22-1.2115.nptl ro root=/dev/hda8
where, instead of "hda8" you have the correct drive and partition where your / partition is. If you make a mistake here, Grub will not complain. Instead, you will get the message saying there was no init found.
8a. If you want to boot in single user mode, without a graphic display, you should add the word single to the end of the line. When the line is complete, press enter.
9. If you do not have a separate /boot partition, type:
Code:

initrd /boot/initrd

If you do have a separate /boot partition, do not enter the /boot and instead, just
Code:

initrd /initrd

In either case, when you hit , Grub will complete the filename, or list possible files ONLY if you have correctly entered all other information. If Grub says something like "no files found", go back and correct your error.
10. Repeat step 9 until you have a complete filename.
11. Press "enter".
12. Finally, if the system doesn't boot automatically, type: boot and press enter.

Kernel Panic's !

2007-12-18T10:13:00.000-08:00

To keep it simple. Kernel panic errors are displayed by the server when it finds itself in a irrecoverable error. The error is mostly seen in linux machines and if you were to compile a new kernel for the machine with wrong settings it is that quick to view :).

Most of the unix like systems or linux systems have a seperate panic() routine which will be responsible for displaying the error output on a console and once this is done, it will be dumping the memory of the kernel on to the disk for further check or debugging. In panic states, you can either restart the machine manually or schedule a automatic reboot.

To set a automatic reboot during kernel panic sessions, use the following settings:

vi /etc/sysctl.conf

now hit i then copy paste this line

kernel.panic = 10

this should initiate a reboot in 10 seconds for normal servers.

A common cause for the kernel panic are memory dump error. The kernel may try accessing a memory area that is currently not available. Again, as mentioned, it can either be due to the a faulty hardware or error with the latest update of the operating system.

Causes for Kernel Panics

* With increasing set of applications or clients wherein for webservers, the increase of RAM becomes inevitable. This is the first and foremost cause for Kernel Panics.

* As mentioned earlier, recompile your kernel and expect it to happen. While recompiling the kernel it is common that we may try few available extensions or support modules which may drive back to this situation unless there is through fixing of the necessary dependencies.

* Hard disk failure, this is common one to happen, your disk may develop a bad sector or bad block

* Odd but a common problem, trouble with system file permissions can drive us to the kernel panic situation.

* Insufficient RAM or Harddisk is another common trouble to be viewed.

* Improperly installed software or hardware.

* Defective hardware like a trouble with CPU

* And very rare chance of having a hardware mismatch which shouldn't happen for most of the branded versions available.

Linux Commands : Kernel administration commands

2007-12-10T05:03:00.000-08:00

Quick commands on linux administration continued...

Kernel administration command,

LSMOD is used to list the loaded modules on the kernel, the usage is:

$lsmod

Module Size Used by
autofs 11904 1 (autoclean)
3c59x 25568 1 (autoclean)
iptable_filter 2288 0 (autoclean) (unused)
md 59008 0 (unused)
usb-uhci 20912 0 (unused)
usbcore 53200 1 [usb-uhci]

**********************************************************************

MODINFO is the command used to learn about the modules active on the kernel.

usage:
$/sbin/modinfo usb-uhci

filename: /lib/modules/2.4.4-18k/kernel/drivers/usb/usb-uhci.o
description: "USB Universal Host Controller Interface driver"
author: "Georg Acher, Deti Fliegl, Thomas Sailer, Roman Weissgaerber"

**********************************************************************

INSMOD is the command used to activate or to add in the command to an active kernel. The syntax is :

$insmod msdos

************************************************************************

RMMOD is the command used to remove any modules that you find running on the server. The list of modules can be found using the command lsmod.

$rmmod msdos

************************************************************************

DEPMOD is the command used to create the dependencies list for the modules. Normally the details are fetched from /etc/modules.conf and have the descriptions added up.

The normal usage of the same is : /sbin/depmod -a

There are other options like depmod -A which should perform the quick check on the dependencies.

*************************************************************************

MODPROBE is another important command that helps the users to load the modules and the dependencies for the functioning. If we are to use

modprobe fat

then it should be loading up the fat module along with the supporting modules. Again,

modprobe -r fat

should remove the module fat along with the supporting modules for the fat.

This should provide you with the best removal technique.

*******************************************************************************

more coming...

quick commands : Linux administration - Hardware section

2007-12-10T03:25:00.000-08:00

Linux will be having most of the hardware devices created at the start of the system but you can always create new devices using the command:

# /dev/MAKEDEV -v ttyS0
create ttyS0 c 4 64 root:dialout 0660

This will be creating a device /dev/ttyS0 with permissions 0660 and ownership as root and dialout as group.

The kernel will be using the major and minor node numbers for the system and the same indicates that the device so created is a serial device.

*************************************************************************

MKNOD is another command used in creating devices. MKDEV is normally the preferred way of creating devices which do not exist but in rare cases there is a chance that the device files may not be known to the MKDEV command.

For the perfect usage of mknod command you should be aware about the minor and major nodes, the file devices.txt should contain the canonical names and details of the nodes.

So for creating the same node as created using the MAKEDEV command the commands will be :

mknod /dev/ttyS0 c 4 24
chmod 0660 /dev/ttyS0
chown root:dialout /dev/ttyS0

*******************************************************************************

LSPCI command is used to list the number of PCI buses available for the server. It is normally used to diagnose the troubles with PCI buses.

usage: $ lspci -tv

*******************************************************************************

LSDEV command is used for learning the information and status of the virtual I/O Devices. The normal usage is:

lsdev -type disk -fields name status

the various types are :

adapter
Lists adapters
disk
Lists disks
lv
Lists logical volumes and volume groups
optical
Lists optical devices (cdrom/dvdrom)
tape
Lists tape devices
tty
Lists tty devices
ent4sea
Lists all physical Ethernet adapters and Etherchannel adapters available for creating a shared Ethernet adapter
ven4sea
Lists all virtual Ethernet adapters available for creating shared Ethernet adapter
ent4ip
Lists all adapters over which interface can be configured.

To display the parent of a devices, type:

lsdev –dev hdisk0 -parent

************************************************************************************

LSUSB is the command used to list the current connected USB connections on the drive.Normal installations may fail to have this command inbuilt you will need to get the packages updated on the distribution with usbutils to get this working.

usage: lsusb

************************************************************************************

lsraid is the command used for displaying the status of the md devices on the server. The details are as mentioned in the configuration file /etc/raidtab. Again the command is able to distinguish between the online and offline devices and can give out the detailed statistics.

usage : lsraid -A

*************************************************************************************

HDPARM is used to modify or view the statistics of the hard disk. The command allows us to pass on the parameters on to the system and also to modify the current disk parameters like DMA settings, power management, etc..

The options available with hdparm can be dangerous on occasions since the parameters if specified incorrectly can cause trouble.

Turn on DMA for the first hard drive.

hdparm -d1 /dev/hda

Turn off DMA for the first hard drive.

hdparm -d0 /dev/hda

*******************************************************************************

more coming up....

LightWeight Directory Access Protocol : LDAP installation configuration and authentication

2007-12-04T23:19:00.000-08:00

Lightweight Directory Access Protocol is used for providing data for the individuals or system users or network devices and the systems in network which require authentication or information from the network. This can be useful for network authentication for critical data or to share address books between the email clients on the network.

LDAP was developed as a protocol to provide centralized and secure access to resources over the network. The earlier standard that was in use was X.500, this was developed towards 1988. X.500 was developed with hierarchial name space and was able to maintain a large quantity of information. There was specific communication link maintained between the directory client and the directory server and the protocol in use was named as Directory Access Protocol.

The X.500 protocol was mainly an application layer protocol and required the whole of OSI level to be installed for successful working. This was the major drawback, since the installation and setting of OSI layers will require high resources and the implementation of directory access protocol at a smaller level of organization will be cost intensive.

LDAP came into existence with protocol much less resource intensive and also had the use of TCP/IP stack compared to the OSI model which was earlier implemented. The LDAP had used few features from the X.500 and was able to remove few resource intensive features from the model.

Installation

The installation of LDAP is mainly with RPMs for redhat based systems and you will need the RPMs:

openldap-2.0.27-8
nss_ldap-202-5
openldap-clients-2.0.27-8
openldap-servers-2.0.27-8
openldap12

normall the redhat systems will work with the above details but some may require these additional packages:

slapd, ldap-utils, libldap2, libldap2-dev

configuration

The major configuration files with LDAP are at:

/etc/openldap/slapd.conf

The configuration contains the details of hostname, domain info, admin info and references.

Sample configuration of slapd.conf

# $OpenLDAP: pkg/ldap/servers/slapd/slapd.conf,v 1.8.8.4 2000/08/26 17:06:18 kurt Exp $
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/inetorgperson.schema
#include /etc/openldap/schema/nis.schema
#include /etc/openldap/schema/redhat/rfc822-MailMember.schema
#include /etc/openldap/schema/redhat/autofs.schema
#include /etc/openldap/schema/redhat/kerberosobject.schema

# Define global ACLs to disable default read access.

# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org

#pidfile /var/run/slapd.pid
#argsfile /var/run/slapd.args

# Load dynamic backend modules:
# modulepath /usr/sbin/openldap
# moduleload back_ldap.la
# moduleload back_ldbm.la
# moduleload back_passwd.la
# moduleload back_shell.la

# To allow TLS-enabled connections, create /usr/share/ssl/certs/slapd.pem
# and uncomment the following lines.
# TLSCertificateFile /usr/share/ssl/certs/slapd.pem
# TLSCertificateKeyFile /usr/share/ssl/certs/slapd.pem

#######################################################################
# ldbm database definitions
#######################################################################

database ldbm
#suffix "dc=stooges,dc=org"
suffix "o=stooges"
rootdn "cn=StoogeAdmin,o=stooges"
rootpw secret1
directory /var/lib/ldap/stooges
defaultaccess read
schemacheck on
lastmod on
#allow *
# Indices to maintain
#index objectClass eq
#index objectClass,uid,uidNumber,gidNumber eq
#index cn,mail,surname,givenname eq,subinitial
index cn,sn,st eq,pres,sub

database ldbm
suffix "o=delta"
# Only one suffix allowed per database
#suffix "dc=delta,dc=org"
rootdn "cn=DeanWormer,o=delta"
rootpw secret2
directory /var/lib/ldap/fraternity
defaultaccess read
schemacheck on
lastmod on
index cn,sn,st eq,pres,sub
--------------------------------------------------

User Management Made Easy

2007-12-04T04:19:00.000-08:00

who command displays the current active users on the server 'w' will still show the result in more compact method
Use who -uH for idle time and terminal info.

users Used to list the currently logged in users.

w Displays currently logged in users and processes they are running.

whoami To display the current username

groups Shows us the list of groups that the current user is in.
Use groups user-id to display groups for a given user.

set Used to define the environment variables normally during installation of softwares we will need to make these custom
settings

id Used to display the user id and the group id's for the user
Use id user-id to display info for another user id.

last Listing of most recent logins by users. Show where from, date and time of login (ftp, ssh, ...) Also see lastlog command.
Show last 100 logins: last -100

history Shell command to display previously entered commands.

If you wish to set a welcome message or warning messages for all SSH connections then it will be advised to add the same to /etc/issue for example and have the /etc/ssh/sshd_config
Specify text file: Banner /etc/issue

Again you can use the /etc/motd to modify the next welcome message which gets displayed on the login prompt. The file /etc/issue is used for welcome message on telnet login screen.

Memory Management: Common commands

2007-12-04T03:53:00.000-08:00

vmstat is used to view the virtual memory usage for the server

cat /proc/meminfo or free is the command used to learn the amount of memory used and the free memory available for the server

pmap is used to understand the shared library and memory map usage.

sar -B is used to know the swaping page statistics on the server.

cat /proc/sys/vm/freepages is used to learn the free virtual memory pages on the system

/usr/bin/time -v date used to learn the page file usage for the programs. This should be used as /usr/bin/time to avoid the regular time command used to display time

/usr/bin/time -v processname

should give the details on paging for the particular program.

cat /proc/filesystems Display filesystems currently in use.

cat /proc/mounts displays the information on the currently mounted file systems on the server.

showmount command will display the current list of network filesystems mounted on to the server.

cat /proc/ide/hda/filename should display the diskinformation of files held by the kernel

mkfs : to create new files systems the general syntax will be:

mkfs -t ext3 /dev/drivename

mount -t ext3 /dev/drivename location

where location should be any part of your filesystem.

fdisk or fsdisk is used for the partitioning of drives on the server. Cfdisk is another partition table manipulator. Each of the commands are equally effective and the difference stays in the switches in use.

"sfdisk" is especially handy for scripting as it has direct support for reading actions from stdin. Again the operations and the flags for the command is different from normal fdisk usage.

Process management commands: Simple and easy to use

2007-12-04T02:29:00.000-08:00

pstree -p The command is used to provide the details on the parent and child processes.

iostat Report the CPU Statistics and the details on the input/output statistics for partitions and devices.

uname -a provides the system information

cat /proc/version displays the kernel version

lsmod To get the details on the currently loaded modules on to the kernel; cat /proc/modules will also show up the same - result.

uptime will show the details of the system updtime.

IPC's : Semaphores, shared memory and queues

Linux uses the semaphores,shared memory and shared memory queues to communicate with each other or for inter process communication.

the common commands used for identifying this are:

ipcs -s lists the semaphores currently in use

ipcs -m shows the shared memory details for the server

ipcs -q shows the shared memory queue

ipcrm - the command used to remove the IPC parameters

ipcrm -s

psof : is used to identify the processes attached to a particular file or network ports

common usages are : lsof pid or file name

lsof - u user-id

netstat -punta

socklist

will provide the list of open network connections which can be further used to gain details on the active ports

lsof -i tcp:port-no

ulimit is another tool which should be helpful in creating user limits for number of processes and the memory access details for the

WebAdminstration Quick Tips: Unix Adminstrative commands

2007-12-02T22:30:00.000-08:00

arp

usage: arp

command used for checking ethernet connectivity and IP address. The command displays the contents based on the results from route command and ifconfig details. Mostly used to get the IP address based on the configuration of network cards.

df

usage: df -h or df -i

Displays the filesystem informaton based on the requirements. -h option displays the information in human readable format and -i option is used for displaying information based on the i nodes usage.

du

usage : du -sch filename/foldername or du -sch *

Displays disk usage information based on the tags, the disk usage can be for a file or a specific directory. It can also be used to get the summary of disk usage on the specific folder or a specific partition based on the location from which the command is ran.

ifconfig

usage: ifconfig or ifconfig ethX:M ipaddress netmask

Ifconfig is normally used to configure the network addresses or to check the working of the current network interfaces on the server. Also you can use the command to add up a specific IP address to a network interface, the process can be alternatively done by editing the file at /etc/sysconfig/network-scripts/ifcfg-ethX file and then restarting the network. Please note that the directory location mentioned is only for redhat based servers, the location may be different in other flavours of linux.

init

usage: init a

where a will be the runlevel number to be started with.

netstat

usage: netstat -nlap

this is used to provide summary on the network usage and the sockets currently active on the server. Again the -r option will be providing the route for particular connections along with the socket details.

nslookup

usage: nslookup domainname

Used to find the IP address information for the particular host and checks the domainname.

ps

usage : ps aux

this is the process usage summary and displays the process id and the status of the process, whether it is in sleep ( S ) state or (R) running or (D) terminated state.

route

usage: route -n

used to find the routing table details on the computer will be effective on the times when there is a connectivity issue. Again using route add and route del commands can be defined to add or remove routing entries. Thank you.

shred

usage: shred -v filename

this will help you remove the files by overwriting the details there. This should be helpful in removing the data without of the fear of retreiving any useful information on this.

top

usage: top -c -d2

the options will help you get detailed list of commands and the operations active on the server and while running the top command you can type the key "h" and have the details sorted out for further use

traceroute

usage: traceroute domainname

this should be helpful to get the detailed route from the source to the destination domainname.

KeepAlive & KeepAliveTimeout : Another key performance upgrades

2007-12-02T21:59:00.000-08:00

KeepAlive

This is the apache directive that helps the users to maintain persistant connections on to the server. This can help in maintaining the consistent performance upgrades for the clients. The value used is

KeepAlive on

this should help in maintaining the connections.

KeepAliveTimeout

KeepAliveTimeout 120

This directive is used to define in seconds on how long the persistant connections on to the server is to be maintained. Once the request is received the directive is used to set the timeout.

MaxKeepAliveRequests,MaxRequestsPerChild, ListenBacklog, MaxSpareServers, MinSpareServers: Some more quick limiting options

2007-12-02T08:41:00.000-08:00

ListenBackLog: Used to define the maximum length of the queue of pending connections.

format:ListenBacklog 100

Normally the settings are tuned fine and should be able to work along with the normal running. But in special cases like constant TCP SYN attacks that are to act on the server, the enty will be effective in controlling the access. Most of the occassions will not require the number to be set out as specified since the operating systems may restrict the number of access using the system call 'Listen' clause and this should be much lesser a number than the current one. Again, there is the factor of operating system and this should change with each running operating system. The number may be higher at some instances that the ones specified in the example.

MaxKeepAliveRequests: Defines the the keep-alive requests per connection made on to the server.

MaxKeepAliveRequests 20

The directive is used to limit the number of keep-alive requests that are to arrive from a particular TCP/IP connection. The situation will be valid when the clause KeepAlive On is set on the server else you will find this function close to no use. 0 is used to specify the unlimited access and it is always advised to use high values to obtain maximum performance.

MaxRequestsPerChild : Defines the Number of connections made by server child process

MaxRequestsPerChild 10000

This limit defines the maximum number of child processes created on the server and after the count the new process will die. When the value is set as 0 then the processes does not die automatically. Again for the keep alive requests only the first request counts for the limit and thus will be equally effective with the maxkeepaliverequests clause.

MaxSpareServers : Defines the number of server child processes on the server.

MaxSpareServers 15

This is the count for the maximum number of idle processes by a server, a server process is counted as idle when it fails to serve any requests. The directive mentions the maximum number of idle processes and beyond this count the parent process will automatically kill the idle process.

MinSpareServers: The minimum count of server child processes.

This count is used to define the minimum number or idle processes on the server. If the server finds the count less than the limit then it will be creating a new process at a minimal rate of 1 per second.

MinSpareServers 10

Maximum number of Clients Limit: MaxClients

2007-12-01T20:16:00.000-08:00

Apache does allow the administrators to limit the amount of requests that are to arrive at the server during any stage of the server run. The keyword MaxClients limits the number of simultaneous requests on to the webserver, by limiting this value the server will prevent the amount of requests and will not create any further requests once the limit is reached.

We can update the limit of 256 clients, for this one must edit the HARD_SERVER_LIMIT entry in httpd.h and recompile Apache.

For the current settings, for all those connections that are to exceed the maxclients limit will be queued up by the kernel as mentioned in the directive: ListenBackLog. This helps back at times when the process is free the server will be able to grant the request.

Resource limit Process count : RLimitNPROC

2007-12-01T19:50:00.000-08:00

As discussed earlier the usage can be limited generally using the said techniques, here is few that should set the limitations and to provide good performance to the server. The tag RLimitNPROC is another useful tool in case of an active shared server. This will be effective in the case of apache servers which does have the suexec enabled, thereby to limit the number of processes active by users or for a specific userID. It accepts one or two arguments, namely they will be the soft and hard limits for the user processes.

RLimitNPROC soft-processes [hard-processes]

eg: RLimitNPROC 2 10

should be ideal for a normal running server with high number of users active on apache server.

It will be accepting the value as 'max' and this will be limiting the amount of processes as defined on the root of the system. Again, this will be a dangerous function to use if the CGI's are to run on the apache userID, and the function will be limiting the number of apache ID's created and the situation may rise where you will find the error message " cannot fork " on the apache server.

So when used wisely this will be a very effective tool on the apache based servers with suexec and also have the use on forums.

Apache Timeout : Optimizations by limiting stale processes

2007-11-30T21:47:00.000-08:00

Timeout is another keyword provided by apache for use, this should help you to limit the stale usage of the apache server under three conditions:

1. Limits the time that the apache processes are to wait for the HTTP GET request.

2. Timeouts between the wait time for PUT or POST activities.

3. The time delays on the ACK signals on requests.

The default timer settings are set to 1200 seconds but this can be altered based on the incoming requests on to the server.

Timeout 60

value should be ideal for an active server.

RLimitCPU : CPU limiting for apache.

2007-11-30T21:26:00.000-08:00

Another Unique tool available with apache which helps to limit the CPU resource usage by apache processes. The values passed on are in seconds or as 'max' which should use the full usage settings as specified on the operating system parameters. The clause takes up one or two arguments based on the minimum and maximum limits that are to be set.

CPU resource limits are expressed in seconds per process.

Again this will be active for all the sub processes from the apache server but will stand invalid for the main activities or the direct child of the apache server.

RLimitCPU 60 120

should be ideal for normal and extreme php queries that are to work with the server. Although you can use the max value for servers as per requirement and can update the limit set by operating system by logging on to the system as root.

Apache Optimization : Simple techniques

2007-11-30T21:02:00.000-08:00

With the increase in the number of clients using the webserver it becomes an important step to optimize the normal webserver to its best to gain performance and to make efficient use of the resources. By default Apache does provide you with a few keywords which when used to its best can yield maximum usage of resources.

1. RLimitMem

Normally the apache server will be using the memory limits specified by the operating system although you will find it comparatively less or at some occasions huge memory limits being used. We will be able to set the memory usage by the apache server on a particular machine using the RLimitMem value set properly. The property is available right from apache 1.2 and is most effective while you find apache server using more server resources that it should be.

The RLimitMem takes up two arguments to define the minimum soft resource limit and the maximum resource limit. It takes up values in bytes or you can use the keyword 'max' which will make it access the maximum memory slot available through the operating system.

RLimitMEM 1048576 2097152

should be ideal for a large server. Please note that once you keep the clause as max and you wish to update the memory usage you will need to login as root or to edit the details on startup phase.

Again, the limit will be applicable to all the processes that are forked out of the main process, this should include the CGI and SSI exec programs, but this limit will stand invalid for main apache processes like logging of stats.

Networking Tools on Windows

2007-11-28T22:18:00.000-08:00

These are a set of command line options available with windows platforms for checking out connectivity and modifying the details there.

Display Connection Configuration: ipconfig /all

Display DNS Cache Info Configuration: ipconfig /displaydns

Clear DNS Cache: ipconfig /flushdns

Release All IP Address Connections: ipconfig /release

Renew All IP Address Connections: ipconfig /renew

Re-Register the DNS connections: ipconfig /registerdns

Change/Modify DHCP Class ID: ipconfig /setclassid

Network Connections: control netconnections

Network Setup Wizard: netsetup.cpl

Test Connectivity: ping www.whatismyip.com

Trace IP address Route: tracert

Displays the TCP/IP protocol sessions: netstat

Display Local Route: route

Display Resolved MAC Addresses: arp

Display Name of Computer Currently on: hostname

Display DHCP Class Information: ipconfig /showclassid

These could be used for instructing the clients or check for the connectivity to the server.

Apache Mod_Rewrite: Basics & rule writing

2007-11-16T08:02:00.000-08:00

Rewrite module one among the powerful tools available with apache to get most of the configurations by-passed and have the sites function like desire. Here are the basics for this:

Expressions in use:

^ represents the start of a string as in case of any other Linux programmes.

$ represents the end of the string or to denote the end of a string.

. Any single character.

(a|b) represents the method for comparison.. i.e, a or b

(a,b,c) Group a section for use

[abc] define the item in range for here it will be a b or c

[^abc] defines the item NOT in range a b or c

a? experssion will count a zero or any one having 'a' as positive results

a* will check for zero or more of a

a+ will check for 1 or more of a

a{3} will check for exactly 3 of a

a{3,6} will check for values between 3 and 6

!(pattern) will check for the details when the patterns do not match.

Redirection Header Code

Here are the normal codes used up for apache to represent errors/warning messages. These can be used in the .htaccess redirection rules to have the required results displayed.

301 redirect permanently
302 redirect permanantly
403 forbidden entries
404 not found
410 Gone

Rewrite Rule Flags
The following set are characters and patterns used for defining rules via .htaccess.

R[=code] redirect to a url using optional coding entries.
F sends out a forbidden header message.
G sends out a "no longer exist" message.
P used to represent proxy entries
L to represent last rule
N to represent next rule or to restart the rules
C to represent the chain of rules
T= is used to define mime types
NS is used to skip the request if it is a internal sub request
NC to define if the entry is case sensitive
QSA to append the query string
NE to define " do not escape" output
PT is known as passthrough
S=x is used to skip x rules
E=var:value used to set environmental variables

Condition Flags
NC - Case sensitive entry
OR - To define the conditional entries.

Format of variables used
Here is the format of the variables available.

%{name of the variable}

The common list of variables in use are:

HTTP_USER_AGENT
HTTP_REFERER
HTTP_COOKIE
HTTP_FORWARDED
HTTP_HOST
HTTP_PROXY_CONNECTION
HTTP_ACCEPT

The details of requests in use are:

REMOTE_ADDR
REMOTE_HOST
REMOTE_USER
REMOTE_IDENT
REQUEST_METHOD
SCRIPT_FILENAME
PATH_INFO
QUERY_STRING
AUTH_TYPE

Server variables available are:

DOCUMENT_ROOT
SERVER_ADMIN
SERVER_NAME
SERVER_ADDR
SERVER_PORT
SERVER_PROTOCOL
SERVER_SOFTWARE

Time variables available are:

TIME_YEAR
TIME_MON
TIME_DAY
TIME_HOUR
TIME_MIN
TIME_SEC
TIME_WDAY
TIME

Special variables in use are:

API_VERSION
THE_REQUEST
REQUEST_URI
REQUEST_FILENAME
IS_SUBREQ

Directives in use are:

RewriteEngine
RewriteOptions
RewriteLog
RewriteLogLevel
RewriteLock
RewriteMap
RewriteBase
RewriteCond
RewriteRule

These are the main variables and details used for defining rules sing apache mod_rewrite to carryout modifications to site without accessing or modifying the main apache configuration

Here are some of the common rules used to define how the variables can be used with rewrite rule:

1. When the site is permanently redirected to a different url:

RewriteCond %{HTTP_HOST} ^www.domainname.com$ [NC]
RewriteRule ^(.*)$ http://www.newdomainname/$1 [R=301,L]

2. Page when moved temporarily:

RewriteRule ^page.htm$ new_page [R, NC, L]

Adding Mod_security with apache

2007-11-15T09:14:00.000-08:00

Adding Mod_Security With Apache2

Mod Security, one of the powerful tools in use for security maintenance in apache servers. The most widely used tool, and with simplest of installations

Visit the url:

https://bsn.breach.com/account/login.php

use the username: Prasul
password : Prasul2004

>> Downloads section you should find the list of softwares available, check the

modsecurity-apache/

directory to find the list of softwares which will include the latest version as well as that of the previous 1.9.x versions. Once you have selected this you can copy the link location and on the shell type the command

wget --no-check-certificate "url u copied without this quotes"

here is what I have done:

cd /usr/local/src

wget --no-check-certificate https://bsn.breach.com/downloads/t=e75bb42feca2d6c5ef356f122d408872/modsecurity-apache/modsecurity-apache_2.1.3.tar.gz

this got me the file named modsecurity-apache_2.1.3.tar.gz which was untarred using:

tar -xzvf modsecurity-apache_2.1.3.tar.gz
cd modsecurity-apache_2.1.3/

here you will find two directories

apache1 and apache 2

based on the configuration and software in use, choose the folder. i am proceeding with apache2 since I have it running up..

cd apache2/

Once you have this updated, check for the module unique_id to be installed. The module should be turned on with the default installation if otherwise, uncomment the entry from your httpd.conf file to have this module enabled and have your apache server restarted.

Now you will need to use the command :

which libxml2

or

locate libxml2 if you do have updatedb running.

Else you should try finding the file at /usr/includes directory.
<<====== LibXML2 installation
If you are missing this you can install the same using the following simple steps:

cd /usr/local/src
wget http://xmlsoft.org/sources/libxml2-2.6.29.tar.gz
wget http://xmlsoft.org/sources/libxslt-1.1.21.tar.gz

tar -xzvf libxml2-2.6.29
cd libxml2-2.6.29/
./configure --prefix=/usr && make
make test
make install
cd ..
tar -xzvf libxslt-1.1.21
cd libxslt-1.1.21
./configure --prefix=/usr && make
make install

you can try running make test before make install to verify the settings but in most occasions you should be good to go with the above steps.

========>>

So now you have the known path to libxml2, enter the modsecurity directory and move along to apache 2 directory there,

cd /usr/local/src/modsecurity-apache_2.1.3/apache2/
pico MakeFile

search and update the location to libxml2 there as:

INCLUDES=-I/usr/include/libxml2

once this is done, save and exit the file and use the command

make
/usr/local/apache2/bin/apachectl stop
make install

pico /usr/local/apache2/conf/httpd.conf

add the entry :

LoadModule security2_module         modules/mod_security2.so

or can use:

LoadModule security_module /usr/lib/httpd/modules/mod_security.so


save and exit the file and start apache.

/usr/local/apache2/bin/apachectl start

now you should have apache up with mod security on.

make sure that you make a good copy of httpd.conf before adding up the rules. The following are the basic rules and for further filtering I will add up to this. Here is something that is basically done:

use IfModule mod_security.c or use the Include clause to have the modules working:

cd /etc/modsecurity/
touch rules.conf
pico /usr/local/apache2/conf/httpd.conf

add the line:

Include /etc/modsecurity/rules.conf

or

Include conf.d/*.conf

save and exit the file. Now restart apache. This should help you work out the configuration in a smooth way. To start with you can use the following set of rules:

pico /etc/mod_security/rules.conf

add the following entries.

# Turn the filtering engine On or Off
SecFilterEngine On

# Make sure that URL encoding is valid
SecFilterCheckURLEncoding On

# Unicode encoding check
SecFilterCheckUnicodeEncoding Off

# Only allow bytes from this range
SecFilterForceByteRange 0 255

# Only log suspicious requests
SecAuditEngine RelevantOnly

# The name of the audit log file
SecAuditLog logs/audit_log
# Debug level set to a minimum
SecFilterDebugLog logs/modsec_debug_log
SecFilterDebugLevel 0

# Should mod_security inspect POST payloads
SecFilterScanPOST On

# By default log and deny suspicious requests
# with HTTP status 500
SecFilterDefaultAction "deny,log,status:500"

""

This should be the basic setup for the modules. I will add few more rules for work.

< "IfModule" >

# Only inspect dynamic requests
# (YOU MUST TEST TO MAKE SURE IT WORKS AS EXPECTED)
#SecFilterEngine DynamicOnly

SecFilterEngine On

# Reject requests with status 500
SecFilterDefaultAction "deny,log,status:500"

# Some sane defaults
SecFilterScanPOST On
SecFilterCheckURLEncoding On
SecFilterCheckCookieFormat On
SecFilterCheckUnicodeEncoding Off
SecFilterNormalizeCookies On
# enable version 1 (RFC 2965) cookies
SecFilterCookieFormat 1

SecServerResponseToken Off

#If you want to scan the output, uncomment these
#SecFilterScanOutput On
#SecFilterOutputMimeTypes "(null) text/html text/plain"

# Accept almost all byte values
SecFilterForceByteRange 1 255

# Server masking is optional
#fake server banner - NOYB used - no one needs to know what we are using
SecServerSignature "NOYB"

#SecUploadDir /tmp
#SecUploadKeepFiles Off

# Only record the interesting stuff
SecAuditEngine RelevantOnly
SecAuditLog logs/audit_log

# You normally won't need debug logging
SecFilterDebugLevel 0
SecFilterDebugLog logs/modsec_debug_log

#And now, the rules
#Remove any of these Include lines you do not use or have rules for.

#First, add in your exclusion rules:
#These MUST come first!
Include /etc/modsecurity/exclude.conf

#Application protection rules
Include /etc/modsecurity/rules.conf

#Comment spam rules
Include /etc/modsecurity/blacklist.conf

#Bad hosts, bad proxies and other bad players
Include /etc/modsecurity/blacklist2.conf

#Bad clients, known bogus useragents and other signs of malware
Include /etc/modsecurity/useragents.conf

#Known bad software, rootkits and other malware
Include /etc/modsecurity/rootkits.conf

#Signatures to prevent proxying through your server
#only rule these rules if your server is NOT a proxy
Include /etc/modsecurity/proxy.conf

#Additional rules for Apache 2.x ONLY!  Do not add this line if you use Apache 1.x
Include /etc/modsecurity/apache2-rules.conf
< / IfModule >

You can also check on the rulesets available at:
http://www.gotroot.com/downloads/ftp/mod_security/apache2/apache2-gotrootrules-
latest.tar.gz

these should work perfectly with apache2.

Apache 2.2.6 Review

2007-11-12T04:30:00.000-08:00

Apache Web Server- Introduction

The word Apache entered English via Spanish, but the ultimate origin is uncertain. The apache project was responsible for producing the most reliable and efficient of free web servers currently available. Because of its simplicity in use and the effectiveness of configurations, apache http server has been able to cover up to 50% of the total active web servers in the world.

The project is currently active under the Apache Software foundation, which is responsible for the development, support and maintenance of the software. The software has been success in large counts and was successful in evolving into the next stage as the version 2 of the basic software. Apache version 2 or the apache2 will be the main point of discussion here. Taking back to the history, the project name was so chosen, since apache represents the name of the American tribe which is best known for their extensive skills in warfare and endurance. As for apache, endurance run has always been the active point of discussion. The project started with codebase of NCSA HTTPd(National Center for Supercomputing Applications webserver, one of the earliest web servers developed by a group of programmers led by Mr. Robert McCool. The NCSA project was later suspended on 1998 and since this happened, apache foundation has removed the relevant codes from the apache web server. Apache server developers have not restricted themselves to a OS platform and has been able to develop the software to its best functionality on UNIX, Linux, Solaris and Mac OS X. They have equal functionalities and procedures available which helps the user to cover up most of the cross platform troubles. Apache software Foundation, in the aim to achieve the best of development has made the software a part of Open Source programs. Thus providing the software for free and still maintaining the best of reputation.

Mr. Robert McCool was responsible for the development for the first version of the software. Mr. Robert had left the NCSA by the mid of 1994 and the code was available for public. On a later stage the new patches and updates for the software were circulated over via emails. Apache had undergone an overhaul with development on the later stages starting from 1995, with a significant effort from Mr. Rob Hartill and crew, they were able to develop new features like pre-forked child processes and later during the development, features like API for better extensibility, pool-based memory allocation, and an adaptive pre-forking process model were added to the software thus making it the best available one. The softwares were subjected to beta testing and addition of standard modules Apache1.0 was launched on December1, 1995.

1.1 Evolution of the server

The development has risen close to the release of Apache 2.2 which includes the maximum changes and updates. Mailing lists stands out as the main source for updates and developments for the project. Although the core members of the foundation still provide their best to the project. Apache 2.2 has made considerable changes to both core program and the list of modules available for the software. Major features like:

==>Bundled authentication and authorization modules have been updated. Mod_authn_alias configurations should provide the best options for the new authentication.

==>Modules like cache, disk_cache, mem_cache has been updated and the new feature by the name htcacheclean has been active for the new version of apache

==>Configuration files are now categorized. You will now have separate configuration files rather than having a single httpd.conf file which was considered to be the central point on earlier versions.

==>Gracefulshutdowntimeout variable has been added to the server to specify the timeout required for the graceful reboot of server.

==>Couple more features like proxies; Regular Expression library has been updated.==>Smart filter module is another important update to the version which allows the firewall to be added with rules that can help filter the results based on the response header or environment variable. It is now able to cover up the dependencies with respect to the earlier release apache2.0

==>Now apache can withstand the 2GB limit which it tend to have in the previous versions. Normally the service used to stop at the point but the latest release will be able to cover files above the limit.

==>Event MPM allows creating separate thread for KEEPALIVE requests and accepting connections. This should help in preventing the timeout delays for accepting new connections.

==>Also introduces new API by the name APR 1.0 API

httpd -M

option is quite remarkable and will let you know the details on the modules loaded on to the current configuration. It has been quite helpful in troubleshooting of most apache errors that you will encounter during the run.

1.2 Installation

Installation instructions for the software will be simple and quite neat.

cd /usr/local/src
wget http://www.uniontransit.com/apache/httpd/httpd-2.2.6.tar.gz
tar -xzvf httpd-2.2.6.tar.gz
cd httpd-2.2.6
./configure --prefix=/usr/local/apache2
make
make install

The compile and installation should be simple as that. Although you will like to use additional options such as

--enable-rewrite=shared FOR ENABLING REWRITE MODULES --enable-speling=shared FOR SPELL CHECK
--enable-so LOADING UP .SO MODULES SPECIFIED
--enable-cgi LOAD UP CGI
--enable-usertrack TRACE BACK APACHE ROUTE
--enable-deflate

 --enable-ssl \
--enable-mime-magic

1.2Configuration

Apache2 presents modularized configuration of sites and functions which should keep the main configuration file free and neat. You will be able to add the virtual host entries to a different file without altering the details of main httpd.conf file. Again, the location doesn't change for the old settings you will still find the apache configuration file at:
pico /usr/local/apache2/conf/httpd.conf

the attributes are similar to what we had for the old versions and to add to this you will have a few more directives which should make the work a better place. So the process of decentralization of configurations has really helped. The configuration settings are same as in previous settings, you will need to use

LoadModule

Or the traditional

“, AddModule, ClearModuleList”

As you will be familiar you will find the following directives still active on the server and you will be able to configure it as per requirements:

Directory
DirectoryMatch
Files
FilesMatch
Location
LocationMatch
VirtualHost

Again you can use the directives like

· AccessFileName

· AllowOverride

As for the function AllowOverride which is mostly confused for the functionality, please make sure that it is entered in the “” tags so that you get the full functionality. You can still get the details of usage at:

http://httpd.apache.org/docs/2.2/mod/core.html#allowoverride

Since the functions are already used with older versions of apache, you should find it quite similar.

"<" VirtualHost [2001:db8::a00:20ff:fea7:ccea]">"
             ServerAdmin webmaster@host.example.com
      DocumentRoot /www/docs/host.example.com
      ServerName host.example.com
      ErrorLog logs/host.example.com-error_log
      TransferLog logs/host.example.com-access_log
    "<"/VirtualHost">"

This will be an addition as it presents to you with the ipv6 format for address based addition of websites.

These are quite noticable changes, although the review part is just complete for the time. But
I am still working with the servers for details. Now for the exact review of the software, the installation worked fine no possible errors were viewed. I had installed this on the apache 1.3 upgraded machine so there was nothing to be added. I will have to try installing on a fresh machine for checks on this regard, meanwhile if you do have trouble with these, then please add comments so that I will keep the records updated.

The cons for the software is it first hand look from 1.3 to 2, when you will view the details you may find it troublesome in getting to the configuration files. You should find this in /usr/local/apache2/conf/sites/
and there are noticable configuration updates there.

As for the security side, w.r.t, to secunia, the release is one of the most stable of configurations and the most secure of available configuration. Currently no hacks have been found to happen with the software and it also has the effective fix for the apache2 ddos vulnerability. Since these are the merits for apache 2.2. I will be constantly updating the thread with the issues am to face. Hope at the end of the day we have a perfect fix or may be a new release.