User Activity, View Processes, Display Memory

1. List Processes based on %CPU and Memory Usage

This script list the processes based on %CPU and Memory usage, with out argument (by default), If you specify the argument (cpu or mem), it lists the processes based on CPU usage or memory usage.

$ vi processes.sh
#! /bin/bash
#List processes based on %cpu and memory usage

echo "Start Time" `date`
# By default, it display the list of processes based on the cpu and memory usage #
if [ $# -eq 0 ]
then

 echo "List of processes based on the %cpu Usage"
 ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu  # sorted based on %cpu
 echo "List of processes based on the memory Usage"
 ps -e -orss=,args= | sort -b -k1,1n # sorted bases rss value

# If arguements are given (mem/cpu)
else
 case "$1" in
 mem)
  echo "List of processes based on the memory Usage"
   ps -e -orss=,args= | sort -b -k1,1n
  ;;
  cpu)
  echo "List of processes based on the %cpu Usage"
  ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu
  ;;
  *)
  echo "Invalid Argument Given \n"
  echo "Usage : $0 mem/cpu"
  exit 1
  esac 

fi
echo "End Time" `date`
exit 0

#chmod +x processes.sh

You can execute the above script as shown below.

$ processes.sh

$ processes.sh mem

$ processes.sh cpu

2. Display Logged in users and who is using high CPU percentage

This script displays few information about the currently logged in users and what they are doing.

$ vi loggedin.sh
#! /bin/bash

w > /tmp/a

echo "Total number of unique users logged in currently"
cat /tmp/a|  sed '1,2d' | awk '{print $1}' | uniq | wc -l
echo ""

echo "List of unique users logged in currently"
cat /tmp/a | sed '1,2d'|  awk '{print $1}' | uniq
echo ""

echo "The user who is using high %cpu"
cat /tmp/a | sed '1,2d' | awk   '$7 > maxuid { maxuid=$7; maxline=$0 }; END { print maxuid, maxline }' 

echo ""
echo "List of users logged in and what they are doing"
cat /tmp/a

 # chmod +x loggedin.sh

 ./loggedin.sh 
Total number of unique users logged in currently
1

List of unique users logged in currently
root

The user who is using high %cpu
0.12s root     pts/0    99.33.7.252      Sat00   16:30m  0.12s  0.12s -bash

List of users logged in and what they are doing
 04:36:49 up 2 days, 20:12,  2 users,  load average: 0.11, 0.05, 0.01
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    99.33.7.252      Sat00   16:30m  0.12s  0.12s -bash
root     pts/1    122.181.129.226  04:17    0.00s  0.02s  0.00s /bin/bash ./log

3. Display Total, Used and Free Memory

The following script displays the total, used and free memory space.

$ vi mem.sh
#! /bin/bash

# Total memory space details

echo "Memory Space Details"
free -t -m | grep "Total" | awk '{ print "Total Memory space : "$2 " MB";
print "Used Memory Space : "$3" MB";
print "Free Memory : "$4" MB";
}'

echo "Swap memory Details"
free -t -m | grep "Swap" | awk '{ print "Total Swap space : "$2 " MB";
print "Used Swap Space : "$3" MB";
print "Free Swap : "$4" MB";
}'

#chmod +x mem.sh
[root@ip-10-202-215-149 ~]# ./mem.sh
Memory Space Details
Total Memory space : 2602 MB
Used Memory Space : 804 MB
Free Memory : 1797 MB
Swap memory Details
Total Swap space : 895 MB
Used Swap Space : 0 MB
Free Swap : 895 MB
[root@ip-10-202-215-149 ~]#

Install Nagios on CentOS 5

• For x86 (32-bit) systems
  • rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
    • From Dag Wieers

• For x64 (64-bit) systems
  • rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
    • From Dag Wieers

Install & Configure Prerequisites

• Install Apache
  • yum install httpd php gcc glibc glibc-common gd gd-devel

• Configure Apache to start on boot
  • /sbin/chkconfig --levels 345 httpd on

• Configure iptables to allow Apache traffic
  • /sbin/iptables -I INPUT -p tcp --dport 80 -j ACCEPT
  • /etc/init.d/iptables save
  • /etc/init.d/iptables restart

Install & Configure Nagios

• Install Nagios & Plugins
  • yum install nagios nagios-plugins nagios-plugins-setuid

• Create the default Nagios web access user & set a password
  • htpasswd -c /etc/nagios/htpasswd.users nagiosadmin

• Verify default config files
  • nagios -v /etc/nagios/nagios.cfg

• Start Nagios
  • service nagios restart

• Start Apache
  • service httpd restart

Verify Install

• Try logging into your new Nagios installation by going to http://servername/nagios/ and logging in with nagiosadmin and the password you set. 

Shutdown Linux Box Automatically Cent OS/RHEL/Fedora

Common Tasks For at Command

1. Shutdown/reboot the system
2. Email yourself files
3. Send birthday remainders

Commands

• at- Execute a task at a specific time. For example, shutdown computer, send birthday remainder etc. Useful to schedule one job at a time or a single future event.
• cron – If you want to shutdown Linux box automatically everyday 8 pm then you need to use cron instead of at command (see below for examples). Useful to schedule recurring events or daily events such as backup data, or check system security etc.

at Command Examples

Let us see how to shutdow a Linux automatically at 8 pm, type the command at 8pm and press [enter] key, then type halt followed by enter key. To save your job press CTRL+D.

# at 8pm


Sample outputs:

at> halt

(Press CTRL+D)
Try out the following utilities:

• atq - List the current at jobs pending.
• atrm - Used to remove pending at jobs.

at command accept fairly complex time specifications, for example:
Run job at 6am on monday:

at 6am monday

Run job in 5 minutes time:

at now + 5 minutes

Run job at 4pm but 3 days later:

at 4pm + 3 days

Run job at 10am on 31st July:

at 10am Jul 31

See the file /usr/share/doc/at/timespec for complete time specifications
and read man pages of at, atq, atrm. Make sure you have atd service 

running, if not start it using the following command: 

# /etc/init.d/atd start

Linux Cron job to Shutdown Linux server/Desktop system

First, login as the root user and at a shell prompt type crontab -e command so that you can add cronjob:

# crontab -e

OR

# crontab -e -u root

Append the following entry to it to shutdown box at 20:00 hrs [24 hour close format]:

0 20 * * * /sbin/shutdown -h now

Save and close the file. 

 

 

Hadoop install on AMIs centos!

cd /etc/yum.repos.d/

[root@ip-10-250-57-221 yum.repos.d]# ll

total 4

-rw-r--r-- 1 root root 2245 Oct 1 2009 CentOS-Base.repo

[root@ip-10-250-57-221 yum.repos.d]# wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo

--2010-07-12 08:47:56-- http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo

Resolving archive.cloudera.com... 184.73.170.21

Connecting to archive.cloudera.com|184.73.170.21|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 211 [text/plain]

Saving to: `cloudera-cdh3.repo'

100%[======================================>] 211 --.-K/s in 0s

2010-07-12 08:47:56 (40.2 MB/s) - `cloudera-cdh3.repo' saved [211/211]

[root@ip-10-250-57-221 yum.repos.d]# ls

CentOS-Base.repo cloudera-cdh3.repo

[root@ip-10-250-57-221 yum.repos.d]#

yum install java-1.6.0-openjdk hadoop-0.20

Loaded plugins: fastestmirror

Determining fastest mirrors

* addons: mirror.fdcservers.net

* base: centos.mirror.nac.net

* extras: mirror.vcu.edu

* updates: mirror.rackspace.com

addons | 951 B 00:00

addons/primary | 202 B 00:00

base | 2.1 kB 00:00

base/primary_db | 1.6 MB 00:00

cloudera-cdh3 | 951 B 00:00

cloudera-cdh3/primary | 18 kB 00:00

cloudera-cdh3 62/62

extras | 2.1 kB 00:00

extras/primary_db | 185 kB 00:06

updates | 1.9 kB 00:00

updates/primary_db | 296 kB 00:00

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package hadoop-0.20.noarch 0:0.20.2+320-1 set to be updated

---> Package java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5 set to be updated

--> Processing Dependency: jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk

--> Processing Dependency: libgif.so.4 for package: java-1.6.0-openjdk

--> Processing Dependency: tzdata-java for package: java-1.6.0-openjdk

--> Running transaction check

---> Package giflib.i386 0:4.1.3-7.1.el5_3.1 set to be updated

---> Package jpackage-utils.noarch 0:1.7.3-1jpp.2.el5 set to be updated

---> Package tzdata-java.i386 0:2010i-1.el5 set to be updated

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================

Package Arch Version Repository Size

================================================================================

Installing:

hadoop-0.20 noarch 0.20.2+320-1 cloudera-cdh3 21 M

java-1.6.0-openjdk i386 1:1.6.0.0-1.11.b16.el5 updates 37 M

Installing for dependencies:

giflib i386 4.1.3-7.1.el5_3.1 base 39 k

jpackage-utils noarch 1.7.3-1jpp.2.el5 base 61 k

tzdata-java i386 2010i-1.el5 updates 176 k

Transaction Summary

================================================================================

Install 5 Package(s)

Update 0 Package(s)

Remove 0 Package(s)

Total download size: 58 M

Is this ok [y/N]: y

Downloading Packages:

(1/5): giflib-4.1.3-7.1.el5_3.1.i386.rpm | 39 kB 00:00

(2/5): jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00

(3/5): tzdata-java-2010i-1.el5.i386.rpm | 176 kB 00:00

(4/5): hadoop-0.20-0.20.2+320-1.noarch.rpm | 21 MB 00:01

(5/5): java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.i386.rpm | 37 MB 00:03

--------------------------------------------------------------------------------

Total 12 MB/s | 58 MB 00:04

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

Installing : giflib 1/5

Installing : jpackage-utils 2/5

Installing : tzdata-java 3/5

Installing : hadoop-0.20 4/5

Installing : java-1.6.0-openjdk 5/5

Installed:

hadoop-0.20.noarch 0:0.20.2+320-1

java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5

Dependency Installed:

giflib.i386 0:4.1.3-7.1.el5_3.1 jpackage-utils.noarch 0:1.7.3-1jpp.2.el5

tzdata-java.i386 0:2010i-1.el5

Complete!

[root@ip-10-250-57-221 yum.repos.d]#

[root@ip-10-250-57-221 conf]# vi hadoop-env.sh

# The java implementation to use. Required.

export JAVA_HOME=/usr/java/jdk1.6.0_14/ (remove # and set JAVA environment)

[root@ip-10-250-57-221 java]# vi ~/.bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

. ~/.bashrc

fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:

JAVA_HOME=/usr/java/jdk1.6.0_14/

export PATH

export JAVA_HOME

unset USERNAME

[root@ip-10-250-57-221 ~]# . .bash_profile

[root@ip-10-250-57-221 ~]# . .bash_profile

[root@ip-10-250-57-221 ~]# . .bash_profile

[root@ip-10-250-57-221 ~]# . .bash_profile

[root@ip-10-250-57-221 ~]# java -version

java version "1.6.0_14"

Java(TM) SE Runtime Environment (build 1.6.0_14-b08)

Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)

[root@ip-10-250-57-221 ~]# echo $JAVA_HOME

/usr/java/jdk1.6.0_14/

[root@ip-10-250-57-221 ~]# echo $PATH

/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:

[root@ip-10-250-57-221 java]# vi ~/.bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

. ~/.bashrc

fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:

JAVA_HOME=/usr/java/jdk1.6.0_14/

HADOOP_HOME=/usr/lib/hadoop-0.20

export PATH

export JAVA_HOME

export HADOOP_HOME

unset USERNAME

echo $HADOOP_HOME

/usr/lib/hadoop-0.20

[root@ip-10-250-57-221 hadoop-0.20]# echo $PATH

/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:

[root@ip-10-250-57-221 conf]# pwd

--> #/usr/lib/hadoop-0.20/conf

--> #conf/core-site.xml:

--> #

--> #

--> # fs.default.name

--> # hdfs://:9000

--> #

--> #

--> #conf/hdfs-site.xml:

--> #

--> #

--> # dfs.replication

--> # 1

--> #

--> #

--> #conf/mapred-site.xml:

--> #

--> #

--> # mapred.job.tracker

--> # :9001

--> #

--> #

Setup passphraseless ssh

Now check that you can ssh to the localhost without a passphrase:

# ssh localhost

If you cannot ssh to localhost without a passphrase, execute the following commands:

# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

# cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

Format a new distributed-filesystem:

# bin/hadoop namenode -format

Start the hadoop daemons:

# bin/start-all.sh

open port 50070 and 50030

Browse the web interface for the NameNode and the JobTracker; by default they are

available at:

NameNode - http://:50070/

JobTracker - http://:50030/