ABHIONLINUX
Site useful for linux administration and web hosting

2013/11/10

Hardening Centos with GrSec Kernel

Kernel hardening is one of the most important security part in linux server management. In this article, I am showing how to install grsec kernel with Centos.

Grsec is powerful linux kernel with security enhancement. Following are the features for grsec kernel.


  • An intelligent and robust Role-Based Access Control (RBAC) system that can generate least privilege policies for your entire system with no configuration
  • Change root (chroot) hardening
  • Extensive auditing
  • /tmp race prevention
  • Protection against exploitable null-pointer dereference bugs in the kernel
  • Reduction of the risk of sensitive information being leaked by arbitrary-read kernel bugs
  • A restriction that allows a user to only view his/her processes
  • Prevention of arbitrary code execution, regardless of the technique used (stack smashing, heap corruption, etc)
  • Prevention of arbitrary code execution in the kernel
  • Randomization of the stack, library, and heap bases
  • Kernel stack base randomization
  • Security alerts and audits that contain the IP address of the person causing the alert

First you need to download the linux kernal that you would like to patch with grsec. You can download linux kernel from www.kernel.org . You can download grsec kernal patch from their website at grsecurity.net

Pre-requisite packages : 
       #  yum groupinstall "Development Tools" -y
       #  yum install ncurses-devel -y

Download latest kernel,  grsec patch and apply the patch


 # wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.2.52.tar.xz
 # wget http://grsecurity.net/stable/grsecurity-2.9.1-3.2.52-201311071633.patch
 # tar -xvf linux-3.2.52.tar.xz
 # cd linux-3.2.52.tar.xz
 # patch -p1 ../grsecurity-2.9.1-3.2.52-201311071633.patch

Compile the patched code and install it. Before compiling, you need to copy the boot config to the source code directort

# cd linux-3.2.52.tar.xz
    # cp /boot/config-`uname -r` .config # make clean && make mrproper # make menuconfig # make bzImage # make modules # make modules_install # make install

Now you can reboot your server into new grsec kernel.




2013/09/13

MySQL server has gone away or MySQL: got a packet bigger than ‘max_allowed_packet’ bytes

While restoring mysql database, you may get this error.

MySQL server has gone away
or
MySQL: got a packet bigger than ‘max_allowed_packet’ bytes

Fix:
You need to increase mysql variables  max_allowed_packet and global net_buffer_length in mysql database


Login is root and set this.

mysql> set global max_allowed_packet=2000000000;
mysql> set global net_buffer_length=2000000;

2013/07/25

iptables rules migration

We have situations where we need to migrate the complete rules in iptables/csf from the old server to the new server. In this case, we can use the command iptables-save

root@help2linux [~]# iptables-save >  iptables_rules_back

This command will write the complete backup of the iptables rules to the file   iptables_rules_back
Now we need to import these rules in to our new server. This can be done using the command
iptables-restore

root@new-help2linux [~]# iptables-restore <  iptables_rules_back


2013/06/29

WHM/cPanel - csf Internal Server Error

While accessing csf through whm, you may get this error.

===============
Internal Server Error
500
No response from subprocess (/usr/local/cpanel/whostmgr/docroot/cgi/addon_csf.cgi): subprocess exited with status 2
cpsrvd/11.38.0.18 Server at xxx.xxx.xxx.xxx
================

Fix: Update the csf version using 
root@server [~]# curl -s configserver.com/free/csupdate | perl

2013/03/09

NginxCP – SyntaxError: ‘yield’ not allowed in a ‘try’ block with a ‘finally’ clause

When you install cpnginx cpanel plugin, you can get the following error.

====================================
root@localhost [/usr/local/src/publicnginx]# ./nginxinstaller install
/usr/local/src/publicnginx
Welcome to the Nginx Admin installer......Starting Install
Generating vhosts...
Traceback (most recent call last):
File "/scripts/createvhosts.py", line 2, in ?
import yaml
File "/usr/lib/python2.4/site-packages/PyYAML-3.10-py2.4-linux-x86_64.egg/yaml/__init__.py", line 26
SyntaxError: 'yield' not allowed in a 'try' block with a 'finally' clause
Installing WHM interface...
deploying booster rockets
Traceback (most recent call last):
File "/usr/local/src/publicnginx/nginxinstaller2", line 9, in ?
import createvhosts
File "/usr/local/src/publicnginx/createvhosts.py", line 2, in ?
import yaml
File "/usr/lib/python2.4/site-packages/PyYAML-3.10-py2.4-linux-x86_64.egg/yaml/__init__.py", line 26
SyntaxError: 'yield' not allowed in a 'try' block with a 'finally' clause
root@localhost [/usr/local/src/publicnginx]#
====================================

You can fix it by running pythonfix

/usr/local/src/publicnginx]# ./pythonfix


2013/03/03

CpNginx as reverse proxy in cpanel servers

We all know that nginx is very faster than apache to handle static pages/image. In this section, I would like to demonstrate to configure nginx as reverse proxy to handle static pages/images and apache will be handling dynamic pages. This will really improve the high traffic server performance.

CpNginx is a cpanel plugin to manage nginx web service in cpanel server. Since nginx is vcry good in procession static files, we will configure nginx to run on port 80 and will forward dynamic request to apache on port 8081. Apache module mod_rpaf is the module that will be helping to configure this. First we need to install mod_rpaf. Following are the steps to install mod_rpaf to apache module directory.
===========

Login as root:
# cd /usr/local/src
# wget http://stderr.net/apache/rpaf/download/mod_rpaf-0.6.tar.gz
# tar xvzf mod_rpaf-0.6.tar.gz
# cd mod_rpaf-0.6
# /usr/local/apache/bin/apxs -i -c -n mod_rpaf-2.0.so mod_rpaf-2.0.c
===========

Following are the steps to install cpnginx in cpanel servers.
============
cd /usr/local/src
wget http://nginxcp.com/latest/nginxadmin.tar
tar xf nginxadmin.tar
cd publicnginx
./nginxinstaller install
=============

Before we install nginx, we need to change the apache port to 8081 from the tweak settings

WHM > Server Configuration > Tweak Settings > Apache non-SSL IP/port

Once this has been done, we need to run the commands given below to permanently save the apache configuration.


#  /usr/local/cpanel/whostmgr/bin/whostmgr2 –updatetweaksettings
#  /usr/local/cpanel/bin/apache_conf_distiller –update –main
#  /scripts/rebuildhttpdconf

Now you need to convert apache virtualhost to nginx virtualhosts. For that you can use the script given below and name it as nginx.sh and give the executable permission. Before that you need to create an empty file /usr/local/nginx/conf/vhost.conf

touch /usr/local/nginx/conf/vhost.conf 

Script to convert apache virtualhost
=============================
#!/bin/sh
 
cat > "/usr/local/nginx/conf/nginx.conf" <
user  nobody;
# no need for more workers in the proxy mode
worker_processes  1;
 
error_log  logs/error.log info;
 
worker_rlimit_nofile  8192;
 
events {
 worker_connections  512; # you might need to increase this setting for busy servers
 use epoll; #  Linux kernels 2.4.x  change to rtsig
}
 
http {
 server_names_hash_max_size 2048;
 
 include    mime.types;
 default_type  application/octet-stream;
 
 sendfile on;
 tcp_nopush on;
 tcp_nodelay on;
 
 keepalive_timeout  10;
 
 gzip on;
 gzip_min_length  1100;
 gzip_buffers  4 32k;
 gzip_types    text/plain application/x-javascript text/xml text/css;
 ignore_invalid_headers on;
 
 client_header_timeout  3m;
 client_body_timeout 3m;
 send_timeout     3m;
 connection_pool_size  256;
 client_header_buffer_size 4k;
 large_client_header_buffers 4 32k;
 request_pool_size  4k;
 output_buffers   4 32k;
 postpone_output  1460;
 
 include "/usr/local/nginx/conf/vhost.conf";
}
 
EOF
 
/bin/cp /dev/null /usr/local/nginx/conf/vhost.conf
 
cd /var/cpanel/users
for USER in *; do
 for DOMAIN in `cat $USER | grep ^DNS | cut -d= -f2`; do
  IP=`cat $USER|grep ^IP|cut -d= -f2`;
  ROOT=`grep ^$USER: /etc/passwd|cut -d: -f6`;
  echo "Converting $DOMAIN for $USER";
 
  cat >> "/usr/local/nginx/conf/vhost.conf" <
   server {
  access_log off;
 
  error_log  logs/vhost-error_log warn;
  listen    80;
  server_name  $DOMAIN www.$DOMAIN;
 
  # uncomment location below to make nginx serve static files instead of Apache
  # NOTE this will cause issues with bandwidth accounting as files wont be logged
  #location ~* \.(gif|jpg|jpeg|png|wmv|avi|mpg|mpeg|mp4|js|css)$ {
  # root   $ROOT/public_html;
  #}
 
  location / {
   proxy_send_timeout   90;
   proxy_read_timeout   90;
 
   proxy_buffer_size    4k;
   # you can increase proxy_buffers here to suppress "an upstream response
   #  is buffered to a temporary file" warning
   proxy_buffers     16 32k;
   proxy_busy_buffers_size 64k;
   proxy_temp_file_write_size 64k;
 
   proxy_connect_timeout 30s;
 
   proxy_redirect  http://www.$DOMAIN:81   http://www.$DOMAIN;
   proxy_redirect  http://$DOMAIN:81   http://$DOMAIN;
 
   proxy_pass   http://$IP:81/;
 
   proxy_set_header   Host   \$host;
   proxy_set_header   X-Real-IP  \$remote_addr;
   proxy_set_header   X-Forwarded-For \$proxy_add_x_forwarded_for;
  }
 }
EOF
 done
done
==================================================

# chmod 755 nginx.sh
# ./nginx.sh

Now you can test the nginx configuration using 

# root@server [~]# /usr/local/sbin/nginx -t
# nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
# nginx: configuration file /etc/nginx/nginx.conf test is successful

now you can start nginx using 
# /etc/init.d/nginx start






2013/03/02

Vzmigrate : vzrst module is not loaded on the destination node


When you try to do live migration of openvz vps using vzmigrate,
you may get this error



==============
[root@server ]# vzmigrate --online 192.162.0.1 101
Starting live migration of CT 1141 to 192.162.0.1
Error: vzrst module is not loaded on the destination node
Error: Can't continue live migration
===============

192.162.0.1 : IP address of the destination hardware node.
101              : VID of the vps that is going to transfer.

Cause : vzrst is not loaded in the destination server which is causing the issue.
You can check the modules loaded in the server using the following command.

[root@server ]# lsmod | grep vz

Fix: 
To this this issue, you need to load the required module. You can load the modules using the following commands

[root@server ]# modprobe vzrst
[root@server ]# modprobe vzcpt


2013/03/01

SSHD Exploit

SSH exploit

We are aware of the recent SSHD expliot that is effecting cpanel with cloudlinux and centos servers. It is also reported that plesk, DA and non RHEL distributions are also effected.

These exploits are done via SSH server.  In 64 bit and 32 bit servers, rootkits will deposit /lib64/libkeyutils.so.1.9 and /lib/libkeyutils.so.1.9 respectively and will change the link /lib64/libkeyutils.so.1 (and /lib/libkeyutils.so.1) to point to that library.

For non-effected servers, it will be as given below.
===============

root@server [~]# ls -ld /lib64/libkeyutils.so*
lrwxrwxrwx 1 root root 18 Apr 23  2012 /lib64/libkeyutils.so.1 -> libkeyutils-1.2.so*
================

This expoit may results in stealing the passwords, ssh key, /etc/shadow , spamming or can act as a backdoor to your server.

Cloudlinux had released the following script to test whether your servers is infected or not.

You can see if your server is infected by running:

$ wget -qq -O - http://www.cloudlinux.com/sshd-hack/check.sh |/bin/bash


To clean up libkeyutils library.
USE IT AT YOUR OWN RISK, THE SCRIPT WASN'T FULLY TESTED
$ wget -qq -O - http://www.cloudlinux.com/sshd-hack/clean.sh |/bin/bash
and reboot the server

There is a chance to re-infect the servers again. In order to prevent this, you will have to block ssh from public and allow only to the IP's that are needed. Also you need to update the passwords and ssh keys which will prevent it from happening again.