Archive for the ‘System Administration’ Category

Enable core dumps with apache, RHEL5

If you are like me, you had some trouble getting apache to dump core on a RHEL5 system. The following steps have produced positive results for me:

echo "ulimit -c unlimited >/dev/null 2>&1" >> /etc/profile
echo "DAEMON_COREFILE_LIMIT='unlimited'" >> /etc/sysconfig/init
echo 1 > /proc/sys/fs/suid_dumpable
echo "core.%p" > /proc/sys/kernel/core_pattern
echo "CoreDumpDirectory /var/apache-core-dumps" > \
/etc/httpd/conf.d/core_dumps.conf
mkdir /var/apache-core-dumps
chown apache: /var/apache-core-dumps
source /etc/profile
/etc/init.d/httpd restart

Now you can test it by sending a SIGSEGV to a random apache child process:

# tail -f /var/log/httpd/error_log | grep -i seg &
# ps auxwww |grep httpd (pick a random pid not owned by root)
# kill -11 2014
# [Mon Jul 06 21:05:39 2009] [notice] child pid 2014 exit signal
Segmentation fault (11), possible coredump in /var/apache-core-dumps
# cd /var/apache-core-dumps
# ls
core.2014

You can then get a backtrace using gdb:

# gdb /usr/sbin/httpd core.2014
(gdb) > bt full

I hope this helps someone. Please feel free to comment.

Redirect domain.com to www.domain.com, universal

I still get allot of these requests, so I thought I would share this. You can add this rewrite rule to the .htaccess file located at the domains documentroot or in the virtualhost or directory context of the apache configuration to achieve the desired result.

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule (.*) http://www.%{HTTP_HOST}$1 [QSA,R=301,L]

Redirect index.php to root of current directory

An SEO conscious customer was having duplicate content issues for:

http://domain.com/something/

and

http://domain.com/something/index.php

Simply rewriting index.php to / will not be effective as httpd makes an internal redirect to the DirectoryIndex, causing a redirect loop.

In order to distinguish between the internal redirect and a client request, the %{THE_REQUEST} server variable is used.

When placed in the in the Directory context in the httpd.conf or in the .htaccess file located in the document root or desired origin directory, the following rule will achieve the desired results:

RewriteEngine On
RewriteBase /
RewriteCond %{THE_REQUEST} ^[A-Z]+\ (.*)/index.php\ HTTP/
RewriteRule .*index.php$ http://%{HTTP_HOST}%1/ [L]

Note, this should be placed before any query_string rewriting you are doing for things like clean urls.

This will work recursively from its origin, so:

http://domain.com/something/index.php

and

http://domain.com/something/else/index.php

will be rewritten to:

http://domain.com/something/

and

http://domain.com/something/else/

Stop the plesk adminstrator from recieving dr.web license failures

… And other things that might annoy him/her

There was a request that went around for months from a plesk administrator griping about the various messages that appear in his mailbox, such as the dr.web license notification. By default, messages to the root and postmaster accounts are sent to the plesk administrator. This is accomplished via qmail aliases.

# cd /var/qmail/alias/
# ls -al
total 28
drwxr-sr-x 2 alias qmail 4096 Mar 9 01:59 .
drwxr-xr-x 11 root qmail 4096 Sep 29 21:14 ..
-rw-r–r– 1 root qmail 26 Dec 10 2007 .qmail-mailer-daemon
-rw-r–r– 1 root qmail 16 Mar 9 01:57 .qmail-postmaster
-rw-r–r– 1 root qmail 16 Mar 9 01:57 .qmail-root

Typically these look something link this:
# cat .qmail-postmaster
&plesk_admin_address@foo.bar.com

To keep the customer happy, I made a slight amendment to the alias:

# cat .qmail-postmaster
|cat >/dev/null
# cat .qmail-root
|cat >/dev/null

Removing the aliases will cause all kinds of noise in the logs and actually break the server’s already fragile rfc compliance (rfc2821:4.5.1) . This allows the messages to be ‘handled’ silently. Since most plesk ‘administrators’ don’t care to know how many failed ssh attempts there were in a month, or how much disk space they are using, it is usually a working solution when you can’t get dr.web to stfu.

tomcat + mod_jk + rhel5

[Warning: This guide needs to be updated. 09.08.2009]

v0.5

Because I’m tired of my peers thinking that the process is difficult, I decided to write a quick guide detailing the process of installing tomcat on rhel5 and configuring mod_jk with apache.

RHEL5 provides tomcat5 via RHN, unless you have access to another secure and maintained repository that provides tomcat, I recommend using RedHat’s package.

To install tomcat:

yum install tomcat5
/etc/init.d/tomcat5 start
chkconfig --levels 345 tomcat5 on

This will install and start the tomcat5 web server on port 8080. The redhat tomcat rpms are nearly identical in implementation to the jpackage rpms. $CATALINA_HOME is still /usr/share/tomcat5. That directory contains symlinks to other places.  So /usr/share/tomcat5/conf is a link to /etc/tomcat5.

To test the installation, I use the tomcats sample webapp which can be acquired and deployed like so:

wget http://tomcat.apache.org/tomcat-5.5-doc/appdev/sample/sample.war
mv sample.war /usr/share/tomcat5/webapps

The cool thing about .war files is the way tomcat interacts with them. Simply moving the sample.war to $CATALINA_HOME/wabapps will deploy the web base application. Tomcat will automatically extract, or explode sample.war to $CATALINA_HOME/wabapps/sample.

You can access the newly deployed application via the tomcat web server.

http://foo.bar:8080/sample/

If you have installed the tomcat5-webapps package, you will notice some other directories populating the webapps directory. There is a special webapp directory called ROOT which represents the root of the tomcat server, which can always be accessed by going to the root of the server.

http://foo.bar:8080/

Unless you plan on running tomcat on port 80, you are going to want apache to hand off, or connect, requests to the tomcat server. This is where mod_jk comes into the picture. mod_jk is an apache module known as a connector.

First, download the correct binary for your architecture and install it:

wget http://archive.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/linux/jk-1.2.23/mod_jk-1.2.23-apache-2.2.x-linux-i686.so

***
For x86_64 change to:

wget http://archive.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/linux/jk-1.2.23/mod_jk-1.2.23-apache-2.2.x-linux-x86_64.so
***

chmod +x mod_jk-1.2.23-apache-2.2.x-linux-i686.so
cp -av mod_jk-1.2.23-apache-2.2.x-linux-i686.so /etc/httpd/modules/mod_jk.so

vim /etc/httpd/conf.d/mod_jk.conf

LoadModule jk_module modules/mod_jk.so

JkWorkersFile /etc/httpd/conf/workers.properties
JkShmFile /var/log/httpd/mod_jk.shm
JkLogFile /var/log/httpd/mod_jk.log
JkLogLevel info


vim /etc/httpd/conf/workers.properties

workers.tomcat_home=/usr/share/tomcat5

workers.java_home=/usr/lib/jvm/java

worker.list=jkworker

worker.jkworker.port=8009
worker.jkworker.host=localhost
worker.jkworker.type=ajp13

Now restart apache:

/etc/init.d/httpd configtest
/etc/init.d/httpd restart

At this point you have everything you need to run jsp pages and servlets with tomcat, while still hanging on to familiar httpd virtual hosting and configuration. If you have not read the tomcat documentation on connectors, then you should at this point.

In order to use mod_jk to connect to a tomcat webapp, we need to mount it via the apache configuration. The JkMount directive takes care of this for us. To demonstrate this run the following:

mkdir /usr/share/tomcat5/webapps/test/

vim /usr/share/tomcat5/webapps/test/index.jsp

Hello, I am a jsp script!

<%
// get client locale
java.util.Locale locale = request.getLocale();

// get Dateformat for client’s locale
java.text.DateFormat dateFormat =
java.text.DateFormat.getDateTimeInstance(
java.text.DateFormat.LONG,
java.text.DateFormat.LONG, locale);

%>

The time is:
<%=dateFormat.format( new java.util.Date() ) %>

Now add the following somewhere in the apache configuration:

JkMount /test/* jkworker

JkMount works similar to the apache alias directive. It can be used in the main configuration or virtualhost context. For testing purposes, you can append the mount line to your mod_jk.conf file.

Restart apache and hit the following:

http://localhost/test/

Tomcat ships with a web based administration webapp and also a manager webapp that can be used to deploy other webapps. To use these features, you need to install them from rhn if they have not already been installaed:

yum install tomcat5-admin-webapps

In order to access these webapps, you need to first authenticate. In tomcat, you can define users and user roles. Users and roles are defined in $CATALINA_HOME/conf/tomcat-users.xml (/etc/tomat5/tomcat-users.xml). The admin webapps require authenticated users to have certain roles before they are granted access. For the admin webapp, a user who has the role ‘admin’ is granted access. For the manager webapp, the user will need the ‘manager’ role.

By default, the admin and manager roles are not created so you must define them and assign a user to them. I typically create an ‘admin’ user and give that user both the admin and manager roles. Open up /etc/tomat5/tomcat-users.xml and edit it to look like this:

<?xml version=’1.0′ encoding=’utf-8′?>
<tomcat-users>
<role rolename=”admin”/>
<role rolename=”tomcat”/>
<role rolename=”manager”/>
<role rolename=”role1″/>
<user username=”admin” password=”SuperSecretPassword” roles=”admin,tomcat,manager”/>
<user username=”tomcat” password=”tomcat” roles=”tomcat”/>
<user username=”both” password=”tomcat” roles=”tomcat,role1″/>
<user username=”role1″ password=”tomcat” roles=”role1″/>
</tomcat-users>

Now you can access both the manager and the admin panel with the admin user. Try this out:

httpd://yourdomain.net:8080/admin
httpd://yourdomain.net:8080/manager

If you don’t want to type the port, you can mount the apps with JkMount /admin/* jkworker or JkMount /manager/* jkworker.

At present, this guide is far from complete. However, I think something like this would have helped me the first time I had to install tomcat on a shared server. Please feel free to post comments and flames.

jrod@blacknode.net

apxs:Error: /usr/bin/apr-config not found!

When installing apc on a fresh rhel4 x86_64 kick, autoconfig failed with the following:

[161472-app1 18:22:17]-(/var/lib/mysql)# pecl install apc
 downloading APC-3.0.18.tgz ...
 Starting to download APC-3.0.18.tgz (115,957 bytes)
 .........................done: 115,957 bytes
 47 source files, building
 running: phpize
 Configuring for:
 PHP Api Version:         20041225
 Zend Module Api No:      20060613
 Zend Extension Api No:   220060519
 1. Use apxs to set compile flags (if using APC with Apache)? : yes
1-1, 'all', 'abort', or Enter to continue: all
 Use apxs to set compile flags (if using APC with Apache)? [yes] :
 1. Use apxs to set compile flags (if using APC with Apache)? : yes
1-1, 'all', 'abort', or Enter to continue:
 building in /var/tmp/pear-build-root/APC-3.0.18
 running: /tmp/pear/temp/APC/configure --with-apxs
 checking for egrep... grep -E
 checking for a sed that does not truncate output... /bin/sed
 checking for gcc... gcc
 checking for C compiler default output file name... a.out
 checking whether the C compiler works... yes
 checking whether we are cross compiling... no
 checking for suffix of executables...
 checking for suffix of object files... o
 checking whether we are using the GNU C compiler... yes
 checking whether gcc accepts -g... yes
 checking for gcc option to accept ANSI C... none needed
 checking whether gcc and cc understand -c and -o together... yes
 checking for system library directory... lib
 checking if compiler supports -R... no
 checking if compiler supports -Wl,-rpath,... yes
 checking build system type... x86_64-redhat-linux-gnu
 checking host system type... x86_64-redhat-linux-gnu
 checking target system type... x86_64-redhat-linux-gnu
 checking for PHP prefix... /usr
 checking for PHP includes... -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib
 checking for PHP extension directory... /usr/lib64/php/modules
 checking for PHP installed headers prefix... /usr/include/php
 checking for re2c... no
 configure: WARNING: You will need re2c 0.12.0 or later if you want to regenerate PHP parsers.
 checking for gawk... gawk
 checking whether apc needs to get compiler flags from apxs...
Sorry, I was not able to successfully run APXS.  Possible reasons:
1.  Perl is not installed;
 2.  Apache was not compiled with DSO support (--enable-module=so);
 3.  'apxs' is not in your path.  Try to use --with-apxs=/path/to/apxs
 The output of /usr/sbin/apxs follows
 Can't exec "/usr/bin/apr-config": No such file or directory at /usr/sbin/apxs line 48.
 Use of uninitialized value in scalar chomp at /usr/sbin/apxs line 53.
 apxs:Error: /usr/bin/apr-config not found!.
 -O2 -g -pipe -m64
 configure: error: Aborting
 ERROR: `/tmp/pear/temp/APC/configure --with-apxs' failed

Solution:

# up2date apr-devel
# up2date apr-util-devel

            

inode 8

Here is a little jewel I came across when attempting to repair a disk with a corrupted journal.

# e2fsck /dev/hdb1
fsck 1.38 (30-Jun-2005)
e2fsck 1.38 (30-Jun-2005)
/dev/hdb1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 8 has illegal block(s). Clear<y>? yes

Illegal block #8559 (1073741824) in inode 8. CLEARED.
Illegal block #8567 (1073741824) in inode 8. CLEARED.
Illegal block #8575 (1073741824) in inode 8. CLEARED.
Illegal block #8583 (1073741824) in inode 8. CLEARED.
Illegal block #8591 (1073741824) in inode 8. CLEARED.
Illegal block #8599 (1073741824) in inode 8. CLEARED.
Illegal block #8607 (1073741824) in inode 8. CLEARED.
Illegal block #8615 (1073741824) in inode 8. CLEARED.
Illegal block #8623 (925040641) in inode 8. CLEARED.
Illegal block #8631 (1073741869) in inode 8. CLEARED.
Illegal block #8639 (1073741824) in inode 8. CLEARED.
Too many illegal blocks in inode 8.
Clear inode<y>? yes

Restarting e2fsck from the beginning…
/dev/hdb1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
….
Continues indefinitely

# tune2fs -f -O  ^has_journal /dev/hdb1
tune2fs 1.38 (30-Jun-2005)
Illegal block number passed to ext2fs_unmark_block_bitmap #1073741824 for block bitmap for /dev/hdb1
Segmentation fault

That led to a quite a chuckle. Thankfully debugfs did not exhibit the same insanity.

# debugfs -w /dev/hdb1
debugfs: feature -has_journal

# fsck -yf /dev/hdb1
# fsck -c /dev/hdb1

# tune2fs -j /dev/hdb1

Inode 8 contains the ext3 journel, so ripping it out allowed the fsck to run.

Installing APC

APC (http://us2.php.net/apc, http://en.wikipedia.org/wiki/PHP_accelerator) is a bytecode caching mechanism for php that greatly increase the performance of php applications. APC is available from pecl and easy to install on most LAMP platforms.


pear channel-update pecl.php.net
mount -o remount,exec /tmp
pecl install apc
mount -o remount,noexec /tmp
echo "extension=apc.so" >/etc/php.d/apc.ini
echo "apc.shm_size=128" >>/etc/php.d/apc.ini

Please keep in mind, apc.ini must reside in a directory that is scanned by php. Also, RHEL3-4 users may have issues updating pear.

On RHEL3, something like this will happen:

pear upgrade pear

pear install something

-bash: pear: command not found

pear will then need to be installed manually:
lynx -source http://pear.php.net/go-pear | php -q

On RHEL4, the issue is that the url had changed to grab packages from pear.php.net. This can be resolved by upgrading to the latest version of pear using the full URL:

pear upgrade http://pear.php.net/get/pear http://pear.php.net/get/dependency

TCP: drop open request from xxx.xxx.xxx.xxx

massive syn-floods can cause this. I have learned some tricks that can help mitigate these attacks.

echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_time
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 0 > /proc/sys/net/ipv4/tcp_sack
echo 1280 > /proc/sys/net/ipv4/tcp_max_syn_backlog
echo 1 > /proc/sys/net/ipv4/tcp_syncookies
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
echo 1 > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses
echo 0 > /proc/sys/net/ipv4/conf/all/accept_redirects
echo 1 > /proc/sys/net/ipv4/conf/all/log_martians
echo 65535 > /proc/sys/net/ipv4/ip_conntrack_max

Useful mysql queries for plesk

Plesk FTP passwords

use psa;
SELECT
domains.displayName,
sys_users.login,
accounts.password
FROM
hosting,
domains,
sys_users,
accounts
WHERE
hosting.dom_id = domains.id
AND
hosting.sys_user_id = sys_users.id
AND
sys_users.account_id = accounts.id
ORDER BY
domains.displayName
;


#get plesk email user's username, pw, and redirect.
use psa; select domains.displayname,mail.mail_name,accounts.password,mail.redir_addr from domains,mail,accounts where domains.id=mail.dom_id and accounts.id=mail.account_id;


Catchalls:


select d.name as
domain, p.value as catchall_address from Parameters p,
DomainServices ds, domains d where d.id = ds.dom_id and
ds.parameters_id = p.id and p.parameter = 'catch_addr' order by d.name


select count(distinct(domains.name)) from domains,DomainServices,Parameters where domains.id=DomainServices.dom_id and DomainServices.parameters_id=Parameters.id and Parameters.value<>"reject";

Return top