This presentation assumes you have never used Amazon Web Services (AWS) in any way. We will step through a tutorial to create an AWS account, login, start a server, and access the running server on the Internet.
Terminology
❖ AWS - Amazon Web Services
❖ EC2 - Elastic Compute Cloud
❖ S3 - Simple Storage Service
Checklist
❖ Access Key ID
❖ Secret Access Key
❖ Account Number
❖ X.509 Private Key File
❖ X.509 Certificate File
Register for EC2/S3
Launching a server
❖ Elastic IP
❖ Elastic Block Store
❖ Custom AMIs
❖ Elastic Fox
❖ S3 Fox
❖ AWS CLI tools
http://sourceforge.net/projects/elasticfox/
http://www.s3fox.net/
Red Hat, Fedora, Gnome, KDE, MySQL, PostgreSQL, PostGIS, Slony, Zarafa, Scalix, SugarCRM, vtiger, CITADEL,OpenOffice, LibreOffice,Wine, Apache, hadoop, Nginx Drupla, Joomla, Jboss, Wordpress, WebGUI, Tomcat, TiKi WiKi, Wikimedia, SpamAssassin, ClamAV, OpenLDAP, OTRS, RT, Samba, Cyrus, Dovecot, Exim, Postfix, sendmail, Amanda, Bacula, DRBD, Heartbeat, Keepalived, Nagios, Zabbix, Zenoss,
Wednesday, August 11, 2010
Tuesday, August 10, 2010
hadoop install on AMIs centos!
cd /etc/yum.repos.d/
[root@ip-10-250-57-221 yum.repos.d]# ll
total 4
-rw-r--r-- 1 root root 2245 Oct 1 2009 CentOS-Base.repo
[root@ip-10-250-57-221 yum.repos.d]# wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
--2010-07-12 08:47:56-- http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
Resolving archive.cloudera.com... 184.73.170.21
Connecting to archive.cloudera.com|184.73.170.21|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 211 [text/plain]
Saving to: `cloudera-cdh3.repo'
100%[======================================>] 211 --.-K/s in 0s
2010-07-12 08:47:56 40.2 MB/s - `cloudera-cdh3.repo' saved [211/211]
[root@ip-10-250-57-221 yum.repos.d]# ls
CentOS-Base.repo cloudera-cdh3.repo
[root@ip-10-250-57-221 yum.repos.d]#
yum install java-1.6.0-openjdk hadoop-0.20
Loaded plugins: fastestmirror
Determining fastest mirrors
addons: mirror.fdcservers.net
base: centos.mirror.nac.net
extras: mirror.vcu.edu
updates: mirror.rackspace.com
addons | 951 B 00:00
addons/primary | 202 B 00:00
base | 2.1 kB 00:00
base/primary_db | 1.6 MB 00:00
cloudera-cdh3 | 951 B 00:00
cloudera-cdh3/primary | 18 kB 00:00
cloudera-cdh3 62/62
extras | 2.1 kB 00:00
extras/primary_db | 185 kB 00:06
updates | 1.9 kB 00:00
updates/primary_db | 296 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package hadoop-0.20.noarch 0:0.20.2+320-1 set to be updated
---> Package java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5 set to be updated
--> Processing Dependency: jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk
--> Processing Dependency: libgif.so.4 for package: java-1.6.0-openjdk
--> Processing Dependency: tzdata-java for package: java-1.6.0-openjdk
--> Running transaction check
---> Package giflib.i386 0:4.1.3-7.1.el5_3.1 set to be updated
---> Package jpackage-utils.noarch 0:1.7.3-1jpp.2.el5 set to be updated
---> Package tzdata-java.i386 0:2010i-1.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
hadoop-0.20 noarch 0.20.2+320-1 cloudera-cdh3 21 M
java-1.6.0-openjdk i386 1:1.6.0.0-1.11.b16.el5 updates 37 M
Installing for dependencies:
giflib i386 4.1.3-7.1.el5_3.1 base 39 k
jpackage-utils noarch 1.7.3-1jpp.2.el5 base 61 k
tzdata-java i386 2010i-1.el5 updates 176 k
Transaction Summary
================================================================================
Install 5 Packages
Update 0 Packages
Remove 0 Packages
Total download size: 58 M
Is this ok [y/N]: y
Downloading Packages:
1/5: giflib-4.1.3-7.1.el5_3.1.i386.rpm | 39 kB 00:00
2/5: jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00
3/5: tzdata-java-2010i-1.el5.i386.rpm | 176 kB 00:00
4/5: hadoop-0.20-0.20.2+320-1.noarch.rpm | 21 MB 00:01
5/5: java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.i386.rpm | 37 MB 00:03
--------------------------------------------------------------------------------
Total 12 MB/s | 58 MB 00:04
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : giflib 1/5
Installing : jpackage-utils 2/5
Installing : tzdata-java 3/5
Installing : hadoop-0.20 4/5
Installing : java-1.6.0-openjdk 5/5
Installed:
hadoop-0.20.noarch 0:0.20.2+320-1
java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5
Dependency Installed:
giflib.i386 0:4.1.3-7.1.el5_3.1 jpackage-utils.noarch 0:1.7.3-1jpp.2.el5
tzdata-java.i386 0:2010i-1.el5
Complete!
[root@ip-10-250-57-221 yum.repos.d]#
[root@ip-10-250-57-221 conf]# vi hadoop-env.sh
# The java implementation to use. Required.
export JAVA_HOME=/usr/java/jdk1.6.0_14/ remove # and set JAVA environment
[root@ip-10-250-57-221 java]# vi ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
export PATH
export JAVA_HOME
unset USERNAME
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# java -version
java version "1.6.0_14"
JavaTM SE Runtime Environment build 1.6.0_14-b08
Java HotSpotTM Client VM build 14.0-b16, mixed mode
[root@ip-10-250-57-221 ~]# echo $JAVA_HOME
/usr/java/jdk1.6.0_14/
[root@ip-10-250-57-221 ~]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
[root@ip-10-250-57-221 java]# vi ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
HADOOP_HOME=/usr/lib/hadoop-0.20
export PATH
export JAVA_HOME
export HADOOP_HOME
unset USERNAME
echo $HADOOP_HOME
/usr/lib/hadoop-0.20
[root@ip-10-250-57-221 hadoop-0.20]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
http://pastie.org/1083764
/usr/lib/hadoop-0.20/conf
conf/core-site.xml:
configuration
property
namefs.default.name/name
valuehdfs://elasticurl:9000/value
/property
/configuration
conf/hdfs-site.xml:
configuration
property
namedfs.replication/name
value1/value
/property
/configuration
conf/mapred-site.xml:
configuration
property
namemapred.job.tracker/name
valueelasticurl:9001/value
/property
/configuration
Setup passphraseless ssh
Now check that you can ssh to the localhost without a passphrase:
# ssh localhost
If you cannot ssh to localhost without a passphrase, execute the following commands:
# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
# cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Format a new distributed-filesystem:
# bin/hadoop namenode -format
Start the hadoop daemons:
# bin/start-all.sh
open port 50070 and 50030
Browse the web interface for the NameNode and the JobTracker; by default they are
available at:
NameNode - http://elasticurl:50070/
JobTracker - http://elasticurl:50030/
[root@ip-10-250-57-221 yum.repos.d]# ll
total 4
-rw-r--r-- 1 root root 2245 Oct 1 2009 CentOS-Base.repo
[root@ip-10-250-57-221 yum.repos.d]# wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
--2010-07-12 08:47:56-- http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
Resolving archive.cloudera.com... 184.73.170.21
Connecting to archive.cloudera.com|184.73.170.21|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 211 [text/plain]
Saving to: `cloudera-cdh3.repo'
100%[======================================>] 211 --.-K/s in 0s
2010-07-12 08:47:56 40.2 MB/s - `cloudera-cdh3.repo' saved [211/211]
[root@ip-10-250-57-221 yum.repos.d]# ls
CentOS-Base.repo cloudera-cdh3.repo
[root@ip-10-250-57-221 yum.repos.d]#
yum install java-1.6.0-openjdk hadoop-0.20
Loaded plugins: fastestmirror
Determining fastest mirrors
addons: mirror.fdcservers.net
base: centos.mirror.nac.net
extras: mirror.vcu.edu
updates: mirror.rackspace.com
addons | 951 B 00:00
addons/primary | 202 B 00:00
base | 2.1 kB 00:00
base/primary_db | 1.6 MB 00:00
cloudera-cdh3 | 951 B 00:00
cloudera-cdh3/primary | 18 kB 00:00
cloudera-cdh3 62/62
extras | 2.1 kB 00:00
extras/primary_db | 185 kB 00:06
updates | 1.9 kB 00:00
updates/primary_db | 296 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package hadoop-0.20.noarch 0:0.20.2+320-1 set to be updated
---> Package java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5 set to be updated
--> Processing Dependency: jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk
--> Processing Dependency: libgif.so.4 for package: java-1.6.0-openjdk
--> Processing Dependency: tzdata-java for package: java-1.6.0-openjdk
--> Running transaction check
---> Package giflib.i386 0:4.1.3-7.1.el5_3.1 set to be updated
---> Package jpackage-utils.noarch 0:1.7.3-1jpp.2.el5 set to be updated
---> Package tzdata-java.i386 0:2010i-1.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
hadoop-0.20 noarch 0.20.2+320-1 cloudera-cdh3 21 M
java-1.6.0-openjdk i386 1:1.6.0.0-1.11.b16.el5 updates 37 M
Installing for dependencies:
giflib i386 4.1.3-7.1.el5_3.1 base 39 k
jpackage-utils noarch 1.7.3-1jpp.2.el5 base 61 k
tzdata-java i386 2010i-1.el5 updates 176 k
Transaction Summary
================================================================================
Install 5 Packages
Update 0 Packages
Remove 0 Packages
Total download size: 58 M
Is this ok [y/N]: y
Downloading Packages:
1/5: giflib-4.1.3-7.1.el5_3.1.i386.rpm | 39 kB 00:00
2/5: jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00
3/5: tzdata-java-2010i-1.el5.i386.rpm | 176 kB 00:00
4/5: hadoop-0.20-0.20.2+320-1.noarch.rpm | 21 MB 00:01
5/5: java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.i386.rpm | 37 MB 00:03
--------------------------------------------------------------------------------
Total 12 MB/s | 58 MB 00:04
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : giflib 1/5
Installing : jpackage-utils 2/5
Installing : tzdata-java 3/5
Installing : hadoop-0.20 4/5
Installing : java-1.6.0-openjdk 5/5
Installed:
hadoop-0.20.noarch 0:0.20.2+320-1
java-1.6.0-openjdk.i386 1:1.6.0.0-1.11.b16.el5
Dependency Installed:
giflib.i386 0:4.1.3-7.1.el5_3.1 jpackage-utils.noarch 0:1.7.3-1jpp.2.el5
tzdata-java.i386 0:2010i-1.el5
Complete!
[root@ip-10-250-57-221 yum.repos.d]#
[root@ip-10-250-57-221 conf]# vi hadoop-env.sh
# The java implementation to use. Required.
export JAVA_HOME=/usr/java/jdk1.6.0_14/ remove # and set JAVA environment
[root@ip-10-250-57-221 java]# vi ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
export PATH
export JAVA_HOME
unset USERNAME
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# . .bash_profile
[root@ip-10-250-57-221 ~]# java -version
java version "1.6.0_14"
JavaTM SE Runtime Environment build 1.6.0_14-b08
Java HotSpotTM Client VM build 14.0-b16, mixed mode
[root@ip-10-250-57-221 ~]# echo $JAVA_HOME
/usr/java/jdk1.6.0_14/
[root@ip-10-250-57-221 ~]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
[root@ip-10-250-57-221 java]# vi ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
HADOOP_HOME=/usr/lib/hadoop-0.20
export PATH
export JAVA_HOME
export HADOOP_HOME
unset USERNAME
echo $HADOOP_HOME
/usr/lib/hadoop-0.20
[root@ip-10-250-57-221 hadoop-0.20]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
http://pastie.org/1083764
/usr/lib/hadoop-0.20/conf
conf/core-site.xml:
configuration
property
namefs.default.name/name
valuehdfs://elasticurl:9000/value
/property
/configuration
conf/hdfs-site.xml:
configuration
property
namedfs.replication/name
value1/value
/property
/configuration
conf/mapred-site.xml:
configuration
property
namemapred.job.tracker/name
valueelasticurl:9001/value
/property
/configuration
Setup passphraseless ssh
# ssh localhost
If you cannot ssh to localhost without a passphrase, execute the following commands:
# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
# cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Format a new distributed-filesystem:
# bin/hadoop namenode -format
Start the hadoop daemons:
# bin/start-all.sh
open port 50070 and 50030
Browse the web interface for the NameNode and the JobTracker; by default they are
available at:
NameNode - http://elasticurl:50070/
JobTracker - http://elasticurl:50030/
JBoss Install on Cent OS AMIs
[root@ip-10-251-81-156 ~]# uname -a
Linux ip-10-251-81-156 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:34:28 EST 2008 x86_64 x86_64
x86_64 GNU/Linux
[root@ip-10-251-81-156 ~]# yum install mysql mysql-server httpd java-1.6.0-openjdk -y
Loaded plugins: fastestmirror
Determining fastest mirrors
* addons: mirror.anl.gov
* base: mirror.cogentco.com
* extras: mirror.cogentco.com
* updates: centos.mirrors.tds.net
addons | 951 B 00:00
addons/primary | 204 B 00:00
base | 2.1 kB 00:00
base/primary_db | 2.1 MB 00:00
extras | 2.1 kB 00:00
extras/primary_db | 226 kB 00:00
updates | 1.9 kB 00:00
updates/primary_db | 346 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.2.3-43.el5.centos set to be updated
addons/filelists | 197 B 00:00
base/filelists_db | 4.0 MB 00:00
extras/filelists_db | 220 kB 00:00
updates/filelists_db | 1.2 MB 00:00
---> Package java-1.6.0-openjdk.x86_64 1:1.6.0.0-1.11.b16.el5 set to be updated
--> Processing Dependency: jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk
--> Processing Dependency: tzdata-java for package: java-1.6.0-openjdk
--> Processing Dependency: libgif.so.4()(64bit) for package: java-1.6.0-openjdk
---> Package mysql.i386 0:5.0.77-4.el5_5.3 set to be updated
--> Processing Dependency: perl(DBI) for package: mysql
---> Package mysql.x86_64 0:5.0.77-4.el5_5.3 set to be updated
---> Package mysql-server.x86_64 0:5.0.77-4.el5_5.3 set to be updated
--> Processing Dependency: perl-DBD-MySQL for package: mysql-server
--> Running transaction check
---> Package giflib.x86_64 0:4.1.3-7.1.el5_3.1 set to be updated
---> Package jpackage-utils.noarch 0:1.7.3-1jpp.2.el5 set to be updated
---> Package perl-DBD-MySQL.x86_64 0:3.0007-2.el5 set to be updated
---> Package perl-DBI.x86_64 0:1.52-2.el5 set to be updated
---> Package tzdata-java.x86_64 0:2010i-1.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================
=========
Package Arch Version Repository Size
=======================================================================
=========
Installing:
httpd x86_64 2.2.3-43.el5.centos base 1.2 M
java-1.6.0-openjdk x86_64 1:1.6.0.0-1.11.b16.el5 updates 35 M
mysql i386 5.0.77-4.el5_5.3 updates 4.8 M
mysql x86_64 5.0.77-4.el5_5.3 updates 4.8 M
mysql-server x86_64 5.0.77-4.el5_5.3 updates 9.8 M
Installing for dependencies:
giflib x86_64 4.1.3-7.1.el5_3.1 base 39 k
jpackage-utils noarch 1.7.3-1jpp.2.el5 base 61 k
perl-DBD-MySQL x86_64 3.0007-2.el5 base 148 k
perl-DBI x86_64 1.52-2.el5 base 600 k
tzdata-java x86_64 2010i-1.el5 updates 176 k
Transaction Summary
=======================================================================
=========
Install 10 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 57 M
Downloading Packages:
(1/10): giflib-4.1.3-7.1.el5_3.1.x86_64.rpm | 39 kB 00:00
(2/10): jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00
(3/10): perl-DBD-MySQL-3.0007-2.el5.x86_64.rpm | 148 kB 00:00
(4/10): tzdata-java-2010i-1.el5.x86_64.rpm | 176 kB 00:00
(5/10): perl-DBI-1.52-2.el5.x86_64.rpm | 600 kB 00:00
(6/10): httpd-2.2.3-43.el5.centos.x86_64.rpm | 1.2 MB 00:00
(7/10): mysql-5.0.77-4.el5_5.3.i386.rpm | 4.8 MB 00:01
(8/10): mysql-5.0.77-4.el5_5.3.x86_64.rpm | 4.8 MB 00:00
(9/10): mysql-server-5.0.77-4.el5_5.3.x86_64.rpm | 9.8 MB 00:00
(10/10): java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.x86_64. | 35 MB 00:02
--------------------------------------------------------------------------------
Total 11 MB/s | 57 MB 00:05
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-DBI 1/10
Installing : mysql 2/10
Installing : perl-DBD-MySQL 3/10
Installing : giflib 4/10
Installing : mysql-server 5/10
Installing : httpd 6/10
Installing : jpackage-utils 7/10
Installing : tzdata-java 8/10
Installing : mysql 9/10
Installing : java-1.6.0-openjdk 10/10
Installed:
httpd.x86_64 0:2.2.3-43.el5.centos
java-1.6.0-openjdk.x86_64 1:1.6.0.0-1.11.b16.el5
mysql.i386 0:5.0.77-4.el5_5.3
mysql.x86_64 0:5.0.77-4.el5_5.3
mysql-server.x86_64 0:5.0.77-4.el5_5.3
Dependency Installed:
giflib.x86_64 0:4.1.3-7.1.el5_3.1 jpackage-utils.noarch 0:1.7.3-1jpp.2.el5
perl-DBD-MySQL.x86_64 0:3.0007-2.el5 perl-DBI.x86_64 0:1.52-2.el5
tzdata-java.x86_64 0:2010i-1.el5
Complete!
[root@ip-10-251-81-156 ~]# vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_16/
export PATH
export JAVA_HOME
unset USERNAME
:wq
[root@ip-10-251-81-156 ~]# . .bash_profile
[root@ip-10-251-81-156 ~]# . .bash_profile
[root@ip-10-251-81-156 ~]# echo $JAVA_HOME
/usr/java/jdk1.6.0_16/
[root@ip-10-251-81-156 ~]# echo $PATH
echo $PATH
/
usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/e
c2/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin:/bin
[root@ip-10-251-81-156 ~]# java -version
java version "1.6.0_14"
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) 64-Bit Server VM (build 14.0-b16, mixed mode)
#mkdir /opt/jboss
[root@ip-10-251-81-156 ~]# wget http://sourceforge.net/projects/jboss/files/JBoss/JBoss-
5.1.0.GA/jboss-5.1.0.GA.zip/download
#unzip jboss-5.1.0.GA.zip -d /opt/jboss
#sh run.sh -b(ifconfig find out what is ipaddress is there for AMIs )
Linux ip-10-251-81-156 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:34:28 EST 2008 x86_64 x86_64
x86_64 GNU/Linux
[root@ip-10-251-81-156 ~]# yum install mysql mysql-server httpd java-1.6.0-openjdk -y
Loaded plugins: fastestmirror
Determining fastest mirrors
* addons: mirror.anl.gov
* base: mirror.cogentco.com
* extras: mirror.cogentco.com
* updates: centos.mirrors.tds.net
addons | 951 B 00:00
addons/primary | 204 B 00:00
base | 2.1 kB 00:00
base/primary_db | 2.1 MB 00:00
extras | 2.1 kB 00:00
extras/primary_db | 226 kB 00:00
updates | 1.9 kB 00:00
updates/primary_db | 346 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.2.3-43.el5.centos set to be updated
addons/filelists | 197 B 00:00
base/filelists_db | 4.0 MB 00:00
extras/filelists_db | 220 kB 00:00
updates/filelists_db | 1.2 MB 00:00
---> Package java-1.6.0-openjdk.x86_64 1:1.6.0.0-1.11.b16.el5 set to be updated
--> Processing Dependency: jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk
--> Processing Dependency: tzdata-java for package: java-1.6.0-openjdk
--> Processing Dependency: libgif.so.4()(64bit) for package: java-1.6.0-openjdk
---> Package mysql.i386 0:5.0.77-4.el5_5.3 set to be updated
--> Processing Dependency: perl(DBI) for package: mysql
---> Package mysql.x86_64 0:5.0.77-4.el5_5.3 set to be updated
---> Package mysql-server.x86_64 0:5.0.77-4.el5_5.3 set to be updated
--> Processing Dependency: perl-DBD-MySQL for package: mysql-server
--> Running transaction check
---> Package giflib.x86_64 0:4.1.3-7.1.el5_3.1 set to be updated
---> Package jpackage-utils.noarch 0:1.7.3-1jpp.2.el5 set to be updated
---> Package perl-DBD-MySQL.x86_64 0:3.0007-2.el5 set to be updated
---> Package perl-DBI.x86_64 0:1.52-2.el5 set to be updated
---> Package tzdata-java.x86_64 0:2010i-1.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================
=========
Package Arch Version Repository Size
=======================================================================
=========
Installing:
httpd x86_64 2.2.3-43.el5.centos base 1.2 M
java-1.6.0-openjdk x86_64 1:1.6.0.0-1.11.b16.el5 updates 35 M
mysql i386 5.0.77-4.el5_5.3 updates 4.8 M
mysql x86_64 5.0.77-4.el5_5.3 updates 4.8 M
mysql-server x86_64 5.0.77-4.el5_5.3 updates 9.8 M
Installing for dependencies:
giflib x86_64 4.1.3-7.1.el5_3.1 base 39 k
jpackage-utils noarch 1.7.3-1jpp.2.el5 base 61 k
perl-DBD-MySQL x86_64 3.0007-2.el5 base 148 k
perl-DBI x86_64 1.52-2.el5 base 600 k
tzdata-java x86_64 2010i-1.el5 updates 176 k
Transaction Summary
=======================================================================
=========
Install 10 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 57 M
Downloading Packages:
(1/10): giflib-4.1.3-7.1.el5_3.1.x86_64.rpm | 39 kB 00:00
(2/10): jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00
(3/10): perl-DBD-MySQL-3.0007-2.el5.x86_64.rpm | 148 kB 00:00
(4/10): tzdata-java-2010i-1.el5.x86_64.rpm | 176 kB 00:00
(5/10): perl-DBI-1.52-2.el5.x86_64.rpm | 600 kB 00:00
(6/10): httpd-2.2.3-43.el5.centos.x86_64.rpm | 1.2 MB 00:00
(7/10): mysql-5.0.77-4.el5_5.3.i386.rpm | 4.8 MB 00:01
(8/10): mysql-5.0.77-4.el5_5.3.x86_64.rpm | 4.8 MB 00:00
(9/10): mysql-server-5.0.77-4.el5_5.3.x86_64.rpm | 9.8 MB 00:00
(10/10): java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.x86_64. | 35 MB 00:02
--------------------------------------------------------------------------------
Total 11 MB/s | 57 MB 00:05
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-DBI 1/10
Installing : mysql 2/10
Installing : perl-DBD-MySQL 3/10
Installing : giflib 4/10
Installing : mysql-server 5/10
Installing : httpd 6/10
Installing : jpackage-utils 7/10
Installing : tzdata-java 8/10
Installing : mysql 9/10
Installing : java-1.6.0-openjdk 10/10
Installed:
httpd.x86_64 0:2.2.3-43.el5.centos
java-1.6.0-openjdk.x86_64 1:1.6.0.0-1.11.b16.el5
mysql.i386 0:5.0.77-4.el5_5.3
mysql.x86_64 0:5.0.77-4.el5_5.3
mysql-server.x86_64 0:5.0.77-4.el5_5.3
Dependency Installed:
giflib.x86_64 0:4.1.3-7.1.el5_3.1 jpackage-utils.noarch 0:1.7.3-1jpp.2.el5
perl-DBD-MySQL.x86_64 0:3.0007-2.el5 perl-DBI.x86_64 0:1.52-2.el5
tzdata-java.x86_64 0:2010i-1.el5
Complete!
[root@ip-10-251-81-156 ~]# vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_16/
export PATH
export JAVA_HOME
unset USERNAME
:wq
[root@ip-10-251-81-156 ~]# . .bash_profile
[root@ip-10-251-81-156 ~]# . .bash_profile
[root@ip-10-251-81-156 ~]# echo $JAVA_HOME
/usr/java/jdk1.6.0_16/
[root@ip-10-251-81-156 ~]# echo $PATH
echo $PATH
/
usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/e
c2/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin:/bin
[root@ip-10-251-81-156 ~]# java -version
java version "1.6.0_14"
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) 64-Bit Server VM (build 14.0-b16, mixed mode)
#mkdir /opt/jboss
[root@ip-10-251-81-156 ~]# wget http://sourceforge.net/projects/jboss/files/JBoss/JBoss-
5.1.0.GA/jboss-5.1.0.GA.zip/download
#unzip jboss-5.1.0.GA.zip -d /opt/jboss
#sh run.sh -b
Resetting windows password using linux livecd
1. Put the livecd in the cd/dvd drive and boot your windows machine from the livecd.
(The application that we are going to use is chntpw. In this example, we will be using fedora/ ubuntu livecd.)
2. Once booted, open the terminal and check for tool named "chntpw". If not there, you can install it using:
3. After the tool is ready, mount the windows partition. Use "fdisk -lu" to check which partition should be mounted. To mount /dev/sda1 (assuming your windows partition is on /dev/sda1), use below command:
4. Find SAM file on the windows partition. Usually it is located in Windows/System32/config. Run the chntpw on the SAM file.
5. Follow the wizard of chntpw and clear administrator or any user's password that you want to access.
6. Save changes and restart machine. You can access the windows without password for administrator and users that you have cleared their password. Make sure you take out the livecd, otherwise the machine will boot into it instead of windows.
chntpw
chntpw version 0.99.6 080526 (sixtyfour), (c) Petter N Hagen
chntpw: change password of a user in a NT/2k/XP/2k3/Vista SAM file, or invoke registry editor.
chntpw [OPTIONS] [systemfile] [securityfile] [otherreghive] [...]
-h This message
-u Username to change, Administrator is default
-l list all users in SAM file
-i Interactive. List users (as -l) then ask for username to change
-e Registry editor. Now with full write support!
-d Enter buffer debugger instead (hex editor),
-t Trace. Show hexdump of structs/segments. (deprecated debug function)
-v Be a little more verbose (for debuging)
-L Write names of changed files to /tmp/changed
-N No allocation mode. Only (old style) same length overwrites possible
(The application that we are going to use is chntpw. In this example, we will be using fedora/ ubuntu livecd.)
2. Once booted, open the terminal and check for tool named "chntpw". If not there, you can install it using:
$ yum install chntpw (if your using ubuntu live cd $ sudo apt-get install chntpw)
3. After the tool is ready, mount the windows partition. Use "fdisk -lu" to check which partition should be mounted. To mount /dev/sda1 (assuming your windows partition is on /dev/sda1), use below command:
$ su mount /dev/sda1 /mnt
4. Find SAM file on the windows partition. Usually it is located in Windows/System32/config. Run the chntpw on the SAM file.
$ cd /mnt/Windows/System32/config
List all user in the SAM record$ chntpw -l SAM
Interactively edit user credential$ chntpw -i SAM
5. Follow the wizard of chntpw and clear administrator or any user's password that you want to access.
6. Save changes and restart machine. You can access the windows without password for administrator and users that you have cleared their password. Make sure you take out the livecd, otherwise the machine will boot into it instead of windows.
chntpw
chntpw version 0.99.6 080526 (sixtyfour), (c) Petter N Hagen
chntpw: change password of a user in a NT/2k/XP/2k3/Vista SAM file, or invoke registry editor.
chntpw [OPTIONS]
-h This message
-u
-l list all users in SAM file
-i Interactive. List users (as -l) then ask for username to change
-e Registry editor. Now with full write support!
-d Enter buffer debugger instead (hex editor),
-t Trace. Show hexdump of structs/segments. (deprecated debug function)
-v Be a little more verbose (for debuging)
-L Write names of changed files to /tmp/changed
-N No allocation mode. Only (old style) same length overwrites possible
Monday, August 9, 2010
CentOS / Red Hat Configure an NTP Client And Server
Install ntp
The ntp package contains utilities and daemons that will synchronize your computer's time to Coordinated Universal Time (UTC) via the NTP protocol and NTP servers. The ntp packageincludes ntpdate (a program for retrieving the date and time from remote machines via a network) and ntpd (a daemon which continuously adjusts system time). Install the ntp package:# yum install ntp
How do I configure an NTP Client?
Simply open /etc/ntp.conf file, enter:# vi /etc/ntp.conf
Make sure the following line exists:
server ntp.server.com
Where,
- ntp.server.com : the hostname or IP address of the site NTP server. If your ntp server located at 192.168.1.5, enter server 192.168.1.5. You can also use public ntp server located at ntp.org.
# echo '30 * * * * root /usr/sbin/ntpd -q -u ntp:ntp' > /etc/cron.d/ntpd
The above instructs crond to run ntpd and after setting the clock just exit, and the -u option instructs it to run as the ntp user.
Configure an NTP Server
If you have lots of server and desktop system, configure your own NTP server. Your NTP server contacts a central NTP server,provided by your ISP or a public timeserver located at ntp.org, to obtain accurate time data. The server then allows other machines on your network to request the time data. Our sample setup:
192.168.1.5 ==> CentOS / Fedora / RHEL NTPD Server. 202.54.1.5 ==> ISP remote NTP server. 192.168.1.0/24 ==> NTP clients including desktop systems.First, install and enable ntpd on 192.168.1.5:
# yum install ntp
# chkconfig ntpd on
Now open /etc/ntp.conf:
# vi /etc/ntp.conf
Make sure the following line exits:
restrict default ignore
Above will deny all access to any machine, server or client. However, you need to specifically authorized policy settings. Set it as follows:
restrict 202.54.1.5 mask 255.255.255.245 nomodify notrap noquery server 202.54.1.5Replace 202.54.1.5 and mask with actual remote ISP or ntp.org NTP server IP. Save and close the file.
Configure NTP clients to access your NTP Server
Now, you need to allow legitimate NTP clients to access the Server. For example, allow 192.168.1.0/24 network to synchronize to this server located at 192.168.1.5. Open /etc/ntp.conf and add policy as follows:# Hosts on local network are less restricted. restrict 192.168.1.0 mask 255.255.255.0 nomodify notrapUpdate your firewall settings, open /etc/sysconfig/iptables.
# vi /etc/sysconfig/iptables
Add the following line, before the final LOG and DROP lines for the RH-Firewall-1-INPUT chain:
-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p udp --dport 123 -j ACCEPTSave and close the file. Finally, start ntpd:
# service ntpd start
# service iptables restart
# netstat -tulpn
NTP to synchronize the system clock
Procedure
Login as the root userType the following command to install ntp
# yum install ntp
Turn on service
# chkconfig ntpd on
Synchronize the system clock with 0.pool.ntp.org server:
# ntpdate pool.ntp.org
Start the NTP:
# /etc/init.d/ntpd start
Tuesday, August 3, 2010
Setting up an SSL secured Webserver with CentOS
1. Getting the required software
For
an SSL encrypted web server you will need a few things. Depending on
your install you may or may not have OpenSSL and mod_ssl, Apache's
interface to OpenSSL. Use yum to get them if you need them.
yum install mod_ssl openssl
Yum will either tell you they are installed or will install them for you.
2. Generate a self-signed certificate
Using
OpenSSL we will generate a self-signed certificate. If you are using
this on a production server you are probably likely to want a key from
Trusted Certificate Authority, but if you are just using this on a
personal site or for testing purposes a self-signed certificate is fine.
To create the key you will need to be root so you can either su to root
or use sudo in front of the commands
# Generate private key openssl genrsa -out ca.key 1024 # Generate CSR openssl req -new -key ca.key -out ca.csr # Generate Self Signed Key openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt # Move the files to the correct locations mv ca.crt /etc/pki/tls/certs mv ca.key /etc/pki/tls/private/ca.key mv ca.csr /etc/pki/tls/private/ca.csr
Then we need to update the Apache SSL configuration file
vi +/SSLCertificateFile /etc/httpd/conf.d/ssl.conf
Change the paths to match where the Key file is stored. If you've used the method above it will be
SSLCertificateFile /etc/pki/tls/certs/ca.crt
Then set the correct path for the Certificate Key File a few lines below. If you've followed the instructions above it is:
SSLCertificateKeyFile /etc/pki/tls/private/ca.key
Quit and save the file and then restart Apache
/etc/init.d/httpd restart
All
being well you should now be able to connect over https to your server
and see a default Centos page. As the certificate is self signed
browsers will generally ask you whether you want to accept the
certificate. Firefox 3 won't let you connect at all but you can override
this.
Restart Apache again using
/etc/init.d/httpd restart
Monday, August 2, 2010
Zabbix installation for Cent OS
Zabbix Installation
We will be installing following components:
*
Zabbix Server (for gathering data)
*
Zabbix Agent (for monitoring)
*
Net-SNMP (for SNMP Support)
*
Jabber (for notifications)
*
OpenIPMI (for monitoring)
*
cURL (for web monitoring)
* For x86 (32-bit) systems
o rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
+ From Dag Wieers
* For x64 (64-bit) systems
o rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
+ From Dag Wieers
First install all the dependencies with yum using the command
# yum install zlib-devel mysql-devel glibc-devel curl-devel gcc automake mysql libidn-devel openssl-devel net-snmp-devel rpm-devel OpenIPMI-devel iksemel iksemel-devel rpm-build checkinstall php-mysql php-gd php-bcmath php-mbstring
wget http://downloads.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/1.8.2/zabbix-1.8.2.tar.gz?use_mirror=freefr
tar -xvf zabbix-1.8.1.tar
# cd zabbix-1.8.1
# ./configure --enable-server --with-mysql --with-net-snmp --with-jabber --with-libcurl --with-openipmi --enable-agent
# make
# checkinstall --nodoc --install=yes –y
**********************************************************************
Done. The new package has been installed and saved to
/usr/src/redhat/RPMS/i386/zabbix-1.8.2-1.i386.rpm
You can remove it from your system anytime using:
rpm -e zabbix-1.8.2-1
**********************************************************************
Creating the MySQL DB
We will have to create a database for Zabbix and than import scripts which will do the rest for us.
First of make sure the mysql daemon is running using
# service mysqld status
If not you can start it with
# service mysqld start
If this is the first time mysql is started, you will probably get a message telling you that root has no password. Change the root password to something safe which you won’t forget with
# mysqladmin –u root password ‘new-password-here’
After the password has been set, its recommended to run the following script in your console to secure your mysql installation
# mysql_secure_installation
Now that we have set a root pw and secured our db we can connect to it, to create a new db for Zabbix.
# mysql – u root –p
and enter the password you set before. Than let us create the db with
mysql> create database zabbix character set utf8;
Next we want to tell mysql that our zabbix user hast he rights to mess with the db so lets give him all rights
mysql> grant all privileges on zabbix.* to ‘zabbix‘@‘localhost‘ identified by ‘set-password-here‘;
mysql> quit;
It's important to remember this password as we a going to need it for the Zabbix configs files. Now we are going to import the sql scripts into the db so we have tables, and data
# cd create/schema
# cat mysql.sql | mysql –u zabbix –p zabbix
# cd ../data
# cat data.sql | mysql –u zabbix –p zabbix
# cat images_mysql.sql | mysql –u zabbix -p zabbix
To make sure the script worked we can list the tables of the Zabbix database to see if any where created.
mysql> use zabbix;
mysql> show tables;
mysql> quit;
Configuring Zabbix
Now lets create a folder for Zabbix where the config files are stored and then copy the appropiet files.
# mkdir /etc/Zabbix
# cd ../..
# cp misc/conf/{zabbix_server.conf,zabbix_agentd.conf} /etc/zabbix/
For the beginning the Zabbix agent config file can be left as how it is, but we do need to tell Zabbix where our database is to be found, for that open /etc/zabbix/zabbix_server.conf with your favourite editor
# vi /etc/zabbix/zabbix_server.conf
Look for DBName, with nano you can do this with Ctrl+W, than type in DBName and press enter. The default value it is set to zabbix, so we can leave that as how it is, but we do need to change the user and password set DBUser to zabbix and DBPassword to the one you set during the mysql configuration above.
DBName=zabbix
DBUser=zabbix
DBPassword=your-zabbix-mysql-password
It is recommended to change the file permissions for the Zabbix config so not every user can just read the password out of our file
# chmod 400 /etc/zabbix/zabbix_server.conf
# chown zabbix /etc/zabbix/zabbix_server.conf
Now Zabbix knows where to connect to and with wich login information
Creating user Zabbix and running Zabbix Server
Zabbix is not ment to be executed by root, so we will have to create a Zabbix user to do the job for us.
# useradd –m –s /bin/bash zabbix
With the parameter –m we tell linux to create a home directory for the user and with –s /bin/bash we tell linux to let the user use the shell bash and were its located.
Now that the user is created we are going to log in with the user with
# su – Zabbix
su means run the shell under the user we specify and the – tells linux to load the environment of the user.
You should be able to recognise from the beginning of the prompt that we are logged in as Zabbix {zabbix@hostname ~}$
Now we can execute the script zabbix_agentd and zabbix_server which are found in /usr/local/sbin/
$ /usr/local/sbin/zabbix_agentd
$ /usr/local/sbin/zabbix_server
if all went well you should get any error messages.
To check if Zabbix is running we can grep the output of all running processes
$ ps -e | grep zabbix
You should see several processe runing as zabbix_server and zabbix_agentd
Its all nice that the server is up and runnin, but latest until the next reboot of the server, Zabbix wont be running anymore. To make the this process run automaticly we will have to add the server and the agent to the system startup sequence.
Creating startup scripts
Zabbix comes with startup scripts but it should be mentioned that this scripts are old. Never the less they still work and all that is needed to be done is one minor changes in the script.
Login in back to root
$ su -
After a successful login let us copy the scripts from our Zabbix source into our /etc/init.d/ folder. this can be done with the following command:
# cd zabbix-1.8.1
# cp misc/init.d/redhat/8.0/zabbix_* /etc/init.d/
this will copy zabbix_server and zabbix_agentd startup scripts.
Open zabbix_server with your favourite editor.
# vi /etc/init.d/zabbix_agentd
and than change the progdir to progdir=”/usr/local/sbin/” do the exactly the same with zabbix_agentd.
Now let us add it to the services with
# chkconfig --add zabbix_server
# chkconfig --add zabbix_agentd
all that is left, it to tell the system on which runlevels it should start them, we are going to use runlevel 3 and 5.
# chkconfig --level 35 zabbix_server on
# chkconfig --level 35 zabbix_agentd on
to verify that the changes have been made, we can use chkconfig to list all configurations with
# chkconfig --list
or to just show the zabbix services we can customize our output with grep
# chkconfig --list | grep zabbix
All that is left to be done, is to make these scripts executable, this is done with the following command
# chmod 755 /etc/init.d/zabbix_server
# chmod 755 /etc/init.d/zabbix_agentd
Now we can control our scripts with
# service zabbix_server {start|stop|restart|condrestart}
Installing the Web frontend
Most Likely Zabbix won't agree with all of your settings and will tell you to change them to the requiered settings. In my case these are all php related problems. Open php.ini with your favourite editor
# vi /etc/php.ini
and change following:
max_execution_time = 600
memory_limit = 256M
post_max_size = 32M
date.timezone = (check on your server which zone your in #date [Rajat@rajat ~]$ date
Mon Aug 2 18:47:02 IST 2010)
mbstring.func_overload = 2
service httpd restart
We have Zabbix Server and Agent installed and running on our server, we can control the them with services and they startup each time the machine boots into runlevel 3 or 5 but we still don’t have our Web frontend.
Once installed its time to copy the web files to the correct folder, with CentOS and Apache this would be /var/www/html/.
# mkdir /var/www/html/zabbix
# cp -r frontends/php/ /var/www/html/
To make the page access able through http://localhost/zabbix we will have to to rename the folder to zabbix.
# mv /var/www/html/php/ /var/www/html/zabbix
Now you should now be able to access the Web frontend trough http://localhost/zabbix
If you encoutner problems try checking if the httpd service is running. If it worked you should see the Welcome screen from Zabbix
Setup Zabbix Agent for Linux
1. wget http://downloads.sourceforge.net/zabbix/zabbix-1.8.2.tar.gz?fromerrorjs=1
2. tar -vxzf zabbix-1.8.2.tar.gz
3. cd zabbix-1.8.2
4. ./configure --enable-agent --prefix=/usr/local/zabbix
5. make install
6. mkdir /etc/zabbix
7. cp misc/conf/zabbix_agentd.conf /etc/zabbix/
8. cp misc/init.d/redhat/8.0/zabbix_agentd /etc/init.d/
9. chmod +x /etc/init.d/zabbix_agentd
10. adduser zabbix
11. chkconfig zabbix_agentd on
12. Edit /etc/zabbix/zabbix_agentd.conf, set Server to your Zabbix Server ip/domain and Hostname to your agent hostname.
13. Edit /etc/init.d/zabbix_agentd, set progdir FROM /usr/local/sbin TO progdir="/usr/local/zabbix/sbin/"
14. /etc/init.d/zabbix_agentd start
15. After than add the host to zabbix server and you can start monitor your agent server now.
We will be installing following components:
*
Zabbix Server (for gathering data)
*
Zabbix Agent (for monitoring)
*
Net-SNMP (for SNMP Support)
*
Jabber (for notifications)
*
OpenIPMI (for monitoring)
*
cURL (for web monitoring)
* For x86 (32-bit) systems
o rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
+ From Dag Wieers
* For x64 (64-bit) systems
o rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
+ From Dag Wieers
First install all the dependencies with yum using the command
# yum install zlib-devel mysql-devel glibc-devel curl-devel gcc automake mysql libidn-devel openssl-devel net-snmp-devel rpm-devel OpenIPMI-devel iksemel iksemel-devel rpm-build checkinstall php-mysql php-gd php-bcmath php-mbstring
wget http://downloads.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/1.8.2/zabbix-1.8.2.tar.gz?use_mirror=freefr
tar -xvf zabbix-1.8.1.tar
# cd zabbix-1.8.1
# ./configure --enable-server --with-mysql --with-net-snmp --with-jabber --with-libcurl --with-openipmi --enable-agent
# make
# checkinstall --nodoc --install=yes –y
**********************************************************************
Done. The new package has been installed and saved to
/usr/src/redhat/RPMS/i386/zabbix-1.8.2-1.i386.rpm
You can remove it from your system anytime using:
rpm -e zabbix-1.8.2-1
**********************************************************************
Creating the MySQL DB
We will have to create a database for Zabbix and than import scripts which will do the rest for us.
First of make sure the mysql daemon is running using
# service mysqld status
If not you can start it with
# service mysqld start
If this is the first time mysql is started, you will probably get a message telling you that root has no password. Change the root password to something safe which you won’t forget with
# mysqladmin –u root password ‘new-password-here’
After the password has been set, its recommended to run the following script in your console to secure your mysql installation
# mysql_secure_installation
Now that we have set a root pw and secured our db we can connect to it, to create a new db for Zabbix.
# mysql – u root –p
and enter the password you set before. Than let us create the db with
mysql> create database zabbix character set utf8;
Next we want to tell mysql that our zabbix user hast he rights to mess with the db so lets give him all rights
mysql> grant all privileges on zabbix.* to ‘zabbix‘@‘localhost‘ identified by ‘set-password-here‘;
mysql> quit;
It's important to remember this password as we a going to need it for the Zabbix configs files. Now we are going to import the sql scripts into the db so we have tables, and data
# cd create/schema
# cat mysql.sql | mysql –u zabbix –p zabbix
# cd ../data
# cat data.sql | mysql –u zabbix –p zabbix
# cat images_mysql.sql | mysql –u zabbix -p zabbix
To make sure the script worked we can list the tables of the Zabbix database to see if any where created.
mysql> use zabbix;
mysql> show tables;
mysql> quit;
Configuring Zabbix
Now lets create a folder for Zabbix where the config files are stored and then copy the appropiet files.
# mkdir /etc/Zabbix
# cd ../..
# cp misc/conf/{zabbix_server.conf,zabbix_agentd.conf} /etc/zabbix/
For the beginning the Zabbix agent config file can be left as how it is, but we do need to tell Zabbix where our database is to be found, for that open /etc/zabbix/zabbix_server.conf with your favourite editor
# vi /etc/zabbix/zabbix_server.conf
Look for DBName, with nano you can do this with Ctrl+W, than type in DBName and press enter. The default value it is set to zabbix, so we can leave that as how it is, but we do need to change the user and password set DBUser to zabbix and DBPassword to the one you set during the mysql configuration above.
DBName=zabbix
DBUser=zabbix
DBPassword=your-zabbix-mysql-password
It is recommended to change the file permissions for the Zabbix config so not every user can just read the password out of our file
# chmod 400 /etc/zabbix/zabbix_server.conf
# chown zabbix /etc/zabbix/zabbix_server.conf
Now Zabbix knows where to connect to and with wich login information
Creating user Zabbix and running Zabbix Server
Zabbix is not ment to be executed by root, so we will have to create a Zabbix user to do the job for us.
# useradd –m –s /bin/bash zabbix
With the parameter –m we tell linux to create a home directory for the user and with –s /bin/bash we tell linux to let the user use the shell bash and were its located.
Now that the user is created we are going to log in with the user with
# su – Zabbix
su means run the shell under the user we specify and the – tells linux to load the environment of the user.
You should be able to recognise from the beginning of the prompt that we are logged in as Zabbix {zabbix@hostname ~}$
Now we can execute the script zabbix_agentd and zabbix_server which are found in /usr/local/sbin/
$ /usr/local/sbin/zabbix_agentd
$ /usr/local/sbin/zabbix_server
if all went well you should get any error messages.
To check if Zabbix is running we can grep the output of all running processes
$ ps -e | grep zabbix
You should see several processe runing as zabbix_server and zabbix_agentd
Its all nice that the server is up and runnin, but latest until the next reboot of the server, Zabbix wont be running anymore. To make the this process run automaticly we will have to add the server and the agent to the system startup sequence.
Creating startup scripts
Zabbix comes with startup scripts but it should be mentioned that this scripts are old. Never the less they still work and all that is needed to be done is one minor changes in the script.
Login in back to root
$ su -
After a successful login let us copy the scripts from our Zabbix source into our /etc/init.d/ folder. this can be done with the following command:
# cd zabbix-1.8.1
# cp misc/init.d/redhat/8.0/zabbix_* /etc/init.d/
this will copy zabbix_server and zabbix_agentd startup scripts.
Open zabbix_server with your favourite editor.
# vi /etc/init.d/zabbix_agentd
and than change the progdir to progdir=”/usr/local/sbin/” do the exactly the same with zabbix_agentd.
Now let us add it to the services with
# chkconfig --add zabbix_server
# chkconfig --add zabbix_agentd
all that is left, it to tell the system on which runlevels it should start them, we are going to use runlevel 3 and 5.
# chkconfig --level 35 zabbix_server on
# chkconfig --level 35 zabbix_agentd on
to verify that the changes have been made, we can use chkconfig to list all configurations with
# chkconfig --list
or to just show the zabbix services we can customize our output with grep
# chkconfig --list | grep zabbix
All that is left to be done, is to make these scripts executable, this is done with the following command
# chmod 755 /etc/init.d/zabbix_server
# chmod 755 /etc/init.d/zabbix_agentd
Now we can control our scripts with
# service zabbix_server {start|stop|restart|condrestart}
Installing the Web frontend
Most Likely Zabbix won't agree with all of your settings and will tell you to change them to the requiered settings. In my case these are all php related problems. Open php.ini with your favourite editor
# vi /etc/php.ini
and change following:
max_execution_time = 600
memory_limit = 256M
post_max_size = 32M
date.timezone = (check on your server which zone your in #date [Rajat@rajat ~]$ date
Mon Aug 2 18:47:02 IST 2010)
mbstring.func_overload = 2
service httpd restart
We have Zabbix Server and Agent installed and running on our server, we can control the them with services and they startup each time the machine boots into runlevel 3 or 5 but we still don’t have our Web frontend.
Once installed its time to copy the web files to the correct folder, with CentOS and Apache this would be /var/www/html/.
# mkdir /var/www/html/zabbix
# cp -r frontends/php/ /var/www/html/
To make the page access able through http://localhost/zabbix we will have to to rename the folder to zabbix.
# mv /var/www/html/php/ /var/www/html/zabbix
Now you should now be able to access the Web frontend trough http://localhost/zabbix
If you encoutner problems try checking if the httpd service is running. If it worked you should see the Welcome screen from Zabbix
Setup Zabbix Agent for Linux
1. wget http://downloads.sourceforge.net/zabbix/zabbix-1.8.2.tar.gz?fromerrorjs=1
2. tar -vxzf zabbix-1.8.2.tar.gz
3. cd zabbix-1.8.2
4. ./configure --enable-agent --prefix=/usr/local/zabbix
5. make install
6. mkdir /etc/zabbix
7. cp misc/conf/zabbix_agentd.conf /etc/zabbix/
8. cp misc/init.d/redhat/8.0/zabbix_agentd /etc/init.d/
9. chmod +x /etc/init.d/zabbix_agentd
10. adduser zabbix
11. chkconfig zabbix_agentd on
12. Edit /etc/zabbix/zabbix_agentd.conf, set Server to your Zabbix Server ip/domain and Hostname to your agent hostname.
13. Edit /etc/init.d/zabbix_agentd, set progdir FROM /usr/local/sbin TO progdir="/usr/local/zabbix/sbin/"
14. /etc/init.d/zabbix_agentd start
15. After than add the host to zabbix server and you can start monitor your agent server now.
User Activity, View Processes, Display Memory
1. List Processes based on %CPU and Memory Usage
This script list the processes based on %CPU and Memory usage, with out argument (by default), If you specify the argument (cpu or mem), it lists the processes based on CPU usage or memory usage.$ vi processes.sh #! /bin/bash #List processes based on %cpu and memory usage echo "Start Time" `date` # By default, it display the list of processes based on the cpu and memory usage # if [ $# -eq 0 ] then echo "List of processes based on the %cpu Usage" ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu # sorted based on %cpu echo "List of processes based on the memory Usage" ps -e -orss=,args= | sort -b -k1,1n # sorted bases rss value # If arguements are given (mem/cpu) else case "$1" in mem) echo "List of processes based on the memory Usage" ps -e -orss=,args= | sort -b -k1,1n ;; cpu) echo "List of processes based on the %cpu Usage" ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu ;; *) echo "Invalid Argument Given \n" echo "Usage : $0 mem/cpu" exit 1 esac fi echo "End Time" `date` exit 0
#chmod +x processes.sh
You can execute the above script as shown below.
$ processes.sh $ processes.sh mem $ processes.sh cpu
2. Display Logged in users and who is using high CPU percentage
This script displays few information about the currently logged in users and what they are doing.$ vi loggedin.sh #! /bin/bash w > /tmp/a echo "Total number of unique users logged in currently" cat /tmp/a| sed '1,2d' | awk '{print $1}' | uniq | wc -l echo "" echo "List of unique users logged in currently" cat /tmp/a | sed '1,2d'| awk '{print $1}' | uniq echo "" echo "The user who is using high %cpu" cat /tmp/a | sed '1,2d' | awk '$7 > maxuid { maxuid=$7; maxline=$0 }; END { print maxuid, maxline }' echo "" echo "List of users logged in and what they are doing" cat /tmp/a
# chmod +x loggedin.sh
./loggedin.sh Total number of unique users logged in currently 1 List of unique users logged in currently root The user who is using high %cpu 0.12s root pts/0 99.33.7.252 Sat00 16:30m 0.12s 0.12s -bash List of users logged in and what they are doing 04:36:49 up 2 days, 20:12, 2 users, load average: 0.11, 0.05, 0.01 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT root pts/0 99.33.7.252 Sat00 16:30m 0.12s 0.12s -bash root pts/1 122.181.129.226 04:17 0.00s 0.02s 0.00s /bin/bash ./log
3. Display Total, Used and Free Memory
The following script displays the total, used and free memory space.$ vi mem.sh #! /bin/bash # Total memory space details echo "Memory Space Details" free -t -m | grep "Total" | awk '{ print "Total Memory space : "$2 " MB"; print "Used Memory Space : "$3" MB"; print "Free Memory : "$4" MB"; }' echo "Swap memory Details" free -t -m | grep "Swap" | awk '{ print "Total Swap space : "$2 " MB"; print "Used Swap Space : "$3" MB"; print "Free Swap : "$4" MB"; }'
#chmod +x mem.sh
[root@ip-10-202-215-149 ~]# ./mem.sh
Memory Space Details
Total Memory space : 2602 MB
Used Memory Space : 804 MB
Free Memory : 1797 MB
Swap memory Details
Total Swap space : 895 MB
Used Swap Space : 0 MB
Free Swap : 895 MB
[root@ip-10-202-215-149 ~]#
Thursday, July 29, 2010
Install Nagios on CentOS 5
- For x86 (32-bit) systems
- For x64 (64-bit) systems
Install & Configure Prerequisites
- Install Apache
yum install httpd php gcc glibc glibc-common gd gd-devel
- Configure Apache to start on boot
/sbin/chkconfig --levels 345 httpd on
- Configure iptables to allow Apache traffic
/sbin/iptables -I INPUT -p tcp --dport 80 -j ACCEPT
/etc/init.d/iptables save
/etc/init.d/iptables restart
Install & Configure Nagios
- Install Nagios & Plugins
yum install nagios nagios-plugins nagios-plugins-setuid
- Create the default Nagios web access user & set a password
htpasswd -c /etc/nagios/htpasswd.users nagiosadmin
- Verify default config files
nagios -v /etc/nagios/nagios.cfg
- Start Nagios
service nagios restart
- Start Apache
service httpd restart
Verify Install
- Try logging into your new Nagios installation by going to
http://servername/nagios/
and logging in withnagiosadmin
and the password you set.
Monday, July 26, 2010
Shutdown Linux Box Automatically Cent OS/RHEL/Fedora
Common Tasks For at Command
- Shutdown/reboot the system
- Email yourself files
- Send birthday remainders
Commands
- at- Execute a task at a specific time. For example, shutdown computer, send birthday remainder etc. Useful to schedule one job at a time or a single future event.
- cron – If you want to shutdown Linux box automatically everyday 8 pm then you need to use cron instead of at command (see below for examples). Useful to schedule recurring events or daily events such as backup data, or check system security etc.
at Command Examples
Let us see how to shutdow a Linux automatically at 8 pm, type the command at 8pm and press [enter] key, then type halt followed by enter key. To save your job press CTRL+D.# at 8pm
Sample outputs:
at> halt
(Press CTRL+D)
Try out the following utilities:
- atq - List the current at jobs pending.
- atrm - Used to remove pending at jobs.
Run job at 6am on monday:
at 6am monday
Run job in 5 minutes time:
at now + 5 minutes
Run job at 4pm but 3 days later:
at 4pm + 3 days
Run job at 10am on 31st July:
at 10am Jul 31
See the file /usr/share/doc/at/timespec for complete time specifications and read man pages of at, atq, atrm. Make sure you have atd service
running, if not start it using the following command:
# /etc/init.d/atd start
Linux Cron job to Shutdown Linux server/Desktop system
First, login as the root user and at a shell prompt type crontab -e command so that you can add cronjob:# crontab -e
OR
# crontab -e -u root
Append the following entry to it to shutdown box at 20:00 hrs [24 hour close format]:
0 20 * * * /sbin/shutdown -h now
Save and close the file.
Thursday, July 22, 2010
Hadoop install on AMIs centos!
cd /etc/yum.repos.d/
[root@ip-10-250-57-221 yum.repos.d]# ll
total 4
-rw-r--r-- 1 root root 2245 Oct 1
2009 CentOS-Base.repo
[root@ip-10-250-57-221 yum.repos.d]#
wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
--2010-07-12 08:47:56--
http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
Resolving archive.cloudera.com...
184.73.170.21
Connecting to
archive.cloudera.com|184.73.170.21|:80... connected.
HTTP request sent, awaiting response...
200 OK
Length: 211 [text/plain]
Saving to: `cloudera-cdh3.repo'
100%[======================================>]
211 --.-K/s in 0s
2010-07-12 08:47:56 (40.2 MB/s) -
`cloudera-cdh3.repo' saved [211/211]
[root@ip-10-250-57-221 yum.repos.d]# ls
CentOS-Base.repo cloudera-cdh3.repo
[root@ip-10-250-57-221 yum.repos.d]#
yum install java-1.6.0-openjdk
hadoop-0.20
Loaded plugins: fastestmirror
Determining fastest mirrors
* addons: mirror.fdcservers.net
* base: centos.mirror.nac.net
* extras: mirror.vcu.edu
* updates: mirror.rackspace.com
addons
| 951 B 00:00
addons/primary
| 202 B 00:00
base
| 2.1 kB 00:00
base/primary_db
| 1.6 MB 00:00
cloudera-cdh3
| 951 B 00:00
cloudera-cdh3/primary
| 18 kB 00:00
cloudera-cdh3
62/62
extras
| 2.1 kB 00:00
extras/primary_db
| 185 kB 00:06
updates
| 1.9 kB 00:00
updates/primary_db
| 296 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package hadoop-0.20.noarch
0:0.20.2+320-1 set to be updated
---> Package java-1.6.0-openjdk.i386
1:1.6.0.0-1.11.b16.el5 set to be updated
--> Processing Dependency:
jpackage-utils >= 1.7.3-1jpp.2 for package: java-1.6.0-openjdk
--> Processing Dependency:
libgif.so.4 for package: java-1.6.0-openjdk
--> Processing Dependency:
tzdata-java for package: java-1.6.0-openjdk
--> Running transaction check
---> Package giflib.i386
0:4.1.3-7.1.el5_3.1 set to be updated
---> Package jpackage-utils.noarch
0:1.7.3-1jpp.2.el5 set to be updated
---> Package tzdata-java.i386
0:2010i-1.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version
Repository Size
================================================================================
Installing:
hadoop-0.20 noarch
0.20.2+320-1 cloudera-cdh3 21 M
java-1.6.0-openjdk i386
1:1.6.0.0-1.11.b16.el5 updates 37 M
Installing for dependencies:
giflib i386
4.1.3-7.1.el5_3.1 base 39 k
jpackage-utils noarch
1.7.3-1jpp.2.el5 base 61 k
tzdata-java i386
2010i-1.el5 updates 176 k
Transaction Summary
================================================================================
Install 5 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 58 M
Is this ok [y/N]: y
Downloading Packages:
(1/5):
giflib-4.1.3-7.1.el5_3.1.i386.rpm | 39 kB 00:00
(2/5):
jpackage-utils-1.7.3-1jpp.2.el5.noarch.rpm | 61 kB 00:00
(3/5): tzdata-java-2010i-1.el5.i386.rpm
| 176 kB 00:00
(4/5):
hadoop-0.20-0.20.2+320-1.noarch.rpm | 21 MB 00:01
(5/5):
java-1.6.0-openjdk-1.6.0.0-1.11.b16.el5.i386.rpm | 37 MB 00:03
--------------------------------------------------------------------------------
Total
12 MB/s | 58 MB 00:04
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : giflib
1/5
Installing : jpackage-utils
2/5
Installing : tzdata-java
3/5
Installing : hadoop-0.20
4/5
Installing : java-1.6.0-openjdk
5/5
Installed:
hadoop-0.20.noarch 0:0.20.2+320-1
java-1.6.0-openjdk.i386
1:1.6.0.0-1.11.b16.el5
Dependency Installed:
giflib.i386 0:4.1.3-7.1.el5_3.1
jpackage-utils.noarch 0:1.7.3-1jpp.2.el5
tzdata-java.i386 0:2010i-1.el5
Complete!
[root@ip-10-250-57-221 yum.repos.d]#
[root@ip-10-250-57-221 conf]# vi
hadoop-env.sh
# The java implementation to use.
Required.
export
JAVA_HOME=/usr/java/jdk1.6.0_14/ (remove # and set JAVA environment)
[root@ip-10-250-57-221 java]# vi
~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup
programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
export PATH
export JAVA_HOME
unset USERNAME
[root@ip-10-250-57-221 ~]# .
.bash_profile
[root@ip-10-250-57-221 ~]# .
.bash_profile
[root@ip-10-250-57-221 ~]# .
.bash_profile
[root@ip-10-250-57-221 ~]# .
.bash_profile
[root@ip-10-250-57-221 ~]# java
-version
java version "1.6.0_14"
Java(TM) SE Runtime Environment (build
1.6.0_14-b08)
Java HotSpot(TM) Client VM (build
14.0-b16, mixed mode)
[root@ip-10-250-57-221 ~]# echo
$JAVA_HOME
/usr/java/jdk1.6.0_14/
[root@ip-10-250-57-221 ~]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
[root@ip-10-250-57-221 java]# vi
~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup
programs
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_14/
HADOOP_HOME=/usr/lib/hadoop-0.20
export PATH
export JAVA_HOME
export HADOOP_HOME
unset USERNAME
echo $HADOOP_HOME
/usr/lib/hadoop-0.20
[root@ip-10-250-57-221 hadoop-0.20]#
echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/ec2/bin:/home/ec2/bin:/root/bin:/home/ec2/bin:/root/bin:/usr/java/default/bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin::/home/ec2/bin:/root/bin:/usr/java/jdk1.6.0_14//bin:
[root@ip-10-250-57-221 conf]# pwd
--> #/usr/lib/hadoop-0.20/conf
--> #conf/core-site.xml:
--> #
--> #
--> # fs.default.name
--> #
hdfs://:9000
--> #
--> #
--> #conf/hdfs-site.xml:
--> #
--> #
--> # dfs.replication
--> # 1
--> #
--> #
--> #conf/mapred-site.xml:
--> #
--> #
--> #
mapred.job.tracker
--> #
:9001
--> #
--> #
Setup passphraseless ssh
Now check that you can ssh to the
localhost without a passphrase:
# ssh localhost
If you cannot ssh to localhost without
a passphrase, execute the following commands:
# ssh-keygen -t dsa -P '' -f
~/.ssh/id_dsa
# cat ~/.ssh/id_dsa.pub >>
~/.ssh/authorized_keys
Format a new distributed-filesystem:
# bin/hadoop namenode -format
Start the hadoop daemons:
# bin/start-all.sh
open port 50070 and 50030
Browse the web interface for the
NameNode and the JobTracker; by default they are
available at:
NameNode - http://:50070/
JobTracker - http://:50030/
Friday, July 9, 2010
TeamCity with LAMP Cent OS AMIs
Installing MySQL 5
To install MySQL, we do this:
yum install mysql mysql-server
Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the
system boots) and start the MySQL server:
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
Run
mysql_secure_installation
to set a password for the user root (otherwise anybody can access your MySQL database!):
[root@server1 ~]# mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none): <-- ENTER
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.
Set root password? [Y/n] <-- ENTER
New password: <-- yourrootsqlpassword
Re-enter new password: <-- yourrootsqlpassword
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] <-- ENTER
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] <-- ENTER
... Success!
By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] <-- ENTER
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] <-- ENTER
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MySQL
installation should now be secure.
Thanks for using MySQL!
Installing Apache2
Apache2 is available as a Fedora package, therefore we can install it like this:
yum install httpd
Now configure your system to start Apache at boot time...
chkconfig --levels 235 httpd on
... and start Apache:
/etc/init.d/httpd start
Now direct your browser to http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/ and you should see
the Apache2 placeholder
page:
(( Optional ))
Installing PHP5
We can install PHP5 and the Apache PHP5 module as follows:
yum install php
We must restart Apache afterwards:
/etc/init.d/httpd restart
vi /var/www/html/info.php
phpinfo();
?>
Now we call that file in a browser (e.g. http://ec2-XX-xx-xxx-xxx.compute-
1.amazonaws.com/info.php):
Getting MySQL Support In PHP5
To get MySQL support in PHP, we can install the php-mysql package. It's a good idea to install some
other PHP5 modules as well as you might need them for your applications. You can search for available
PHP5 modules like this:
yum search php
Pick the ones you need and install them like this:
yum install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-
eaccelerator php-magickwand php-magpierss php-mbstring php-mcrypt php-mssql php-shout php-
snmp php-soap php-tidy
Now restart Apache2:
/etc/init.d/httpd restart
Now reload http://ec2-xXX-xX-xXx-XXx.compute-1.amazonaws.com/info.php in your browser and scroll
down to the modules section again. You should now find lots of new modules there, including the
MySQL module:
phpMyAdmin
phpMyAdmin is a web interface through which you can manage your MySQL databases.
phpMyAdmin can be installed as follows:
yum install phpmyadmin
Now we configure phpMyAdmin. We change the Apache configuration so that phpMyAdmin allows
connections not just from localhost (by commenting out the
stanza):
vi /etc/httpd/conf.d/phpMyAdmin.conf
# phpMyAdmin - Web based MySQL browser written in php
#
# Allows only localhost by default
#
# But allowing phpMyAdmin to anyone other than localhost should be considered
# dangerous unless properly secured by SSL
Alias /phpMyAdmin /usr/share/phpMyAdmin
Alias /phpmyadmin /usr/share/phpMyAdmin
#
# order deny,allow
# deny from all
# allow from 127.0.0.1
# allow from ::1
#
# This directory does not require access over HTTP - taken from the original
# phpMyAdmin upstream tarball
#
Order Deny,Allow
Deny from All
Allow from None
# This configuration prevents mod_security at phpMyAdmin directories from
# filtering SQL etc. This may break your mod_security implementation.
#
#
#
# SecRuleInheritance Off
#
#
Restart Apache:
/etc/init.d/httpd restart
Afterwards, you can access phpMyAdmin under http://ec2-XXX-XX-XXX-XXX.compute-
1.amazonaws.com/phpmyadmin/:
Red Hat Enterprise Linux, CentOS, etc.
The OpenJDK 6 packages are also available in EPEL, a community-run project which makes Fedora
packages available to users of Red Hat Enterprise Linux 5, CentOS 5, and other RHEL 5 derivatives.
First install the package that enables the EPEL repository:
$ su -c "rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm"
If installing from a live image, such as a live CD or USB flash drive, add the OpenJDK 6 runtime after
the install:
* To install with the graphical package manager, follow Applications > Add/Remove Software and
use search to find the package(s) to install.
* To install from the command line:
$ su -c "yum install java-1.6.0-openjdk"
This package contains just the Java Runtime Environment. If you want to develop Java programs
then install the java-1.6.0-openjdk-devel package. You can also install all the OpenJDK 6 packages,
including the API documentation, by using the wildcard java-1.6.0-openjdk*.
On Fedora 8 the OpenJDK 7 runtime and development packages are installed by default during any
large-media install, such as from the Fedora 8 DVD, under the name IcedTea. If installing from a live
image then follow the above instructions but replace 1.6.0-openjdk with 1.7.0-icedtea.
yum install tomcat5 tomcat5-webapps tomcat5-admin-webapps
Set JAVA_HOME / PATH for single user
Login to your account and open .bash_profile file
$ vi ~/.bash_profile
Set JAVA_HOME as follows using syntax export JAVA_HOME=. If your path is set
to /usr/java/jdk1.5.0_07/bin/java, set it as follows:
export JAVA_HOME=/usr/java/jdk1.6.0_16/bin/java
Set PATH as follows:
export PATH=$PATH:/usr/java/jdk1.6.0_16/bin
Save and close the file. Just logout and login back to see new changes:
$ echo $JAVA_HOME
$ echo $PATH
Edit the .bash_profile for Java Environment
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_16/
export CATALINA_HOME=/opt/jetbrains/TeamCity
export PATH
export JAVA_HOME
unset USERNAME
http://www.jetbrains.com/teamcity/
Download TeamCity 5.1.2
http://download.jetbrains.com/teamcity/TeamCity-5.1.2.tar.gz
# cd /opt
# mkdir jetbrains
# cd jetbrains
# mkdir TeamCity
# cp TeamCity-5.1.2.tar.gz /opt/jetbrains/TeamCity
# vi /opt/jetnrains/TeamCity/conf/server.xml
connectionTimeout="20000"
redirectPort="8443"
enableLookup="false"
useBodyEncodingForURI="true"
/>
8543 will default port change to 8443
# service httpd restart
# chkconfig httpd on
# service mysqld restart
# chkconfig mysqld on
# service tomcat5 restart
# chkconfig tomcat5 on
# cd /opt/jetbrains/TeamCity/bin
TeamCity server can be started and stopped by the scripts provided in the/bin
directory
To start/stop TeamCity server and default agent at the same time, use the runAll script.
To start/stop only the TeamCity server, use teamcity-server script.
For example:
* Use runAll.bat start to start the server and the default agent
* Use runAll.bat stop to stop the server and the default agent
By default, TeamCity runs on http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:8111/ and has
one registered build agent that runs on the same computer.
To install MySQL, we do this:
yum install mysql mysql-server
Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the
system boots) and start the MySQL server:
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
Run
mysql_secure_installation
to set a password for the user root (otherwise anybody can access your MySQL database!):
[root@server1 ~]# mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none): <-- ENTER
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.
Set root password? [Y/n] <-- ENTER
New password: <-- yourrootsqlpassword
Re-enter new password: <-- yourrootsqlpassword
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] <-- ENTER
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] <-- ENTER
... Success!
By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] <-- ENTER
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] <-- ENTER
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MySQL
installation should now be secure.
Thanks for using MySQL!
Installing Apache2
Apache2 is available as a Fedora package, therefore we can install it like this:
yum install httpd
Now configure your system to start Apache at boot time...
chkconfig --levels 235 httpd on
... and start Apache:
/etc/init.d/httpd start
Now direct your browser to http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/ and you should see
the Apache2 placeholder
page:
(( Optional ))
Installing PHP5
We can install PHP5 and the Apache PHP5 module as follows:
yum install php
We must restart Apache afterwards:
/etc/init.d/httpd restart
vi /var/www/html/info.php
phpinfo();
?>
Now we call that file in a browser (e.g. http://ec2-XX-xx-xxx-xxx.compute-
1.amazonaws.com/info.php):
Getting MySQL Support In PHP5
To get MySQL support in PHP, we can install the php-mysql package. It's a good idea to install some
other PHP5 modules as well as you might need them for your applications. You can search for available
PHP5 modules like this:
yum search php
Pick the ones you need and install them like this:
yum install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-
eaccelerator php-magickwand php-magpierss php-mbstring php-mcrypt php-mssql php-shout php-
snmp php-soap php-tidy
Now restart Apache2:
/etc/init.d/httpd restart
Now reload http://ec2-xXX-xX-xXx-XXx.compute-1.amazonaws.com/info.php in your browser and scroll
down to the modules section again. You should now find lots of new modules there, including the
MySQL module:
phpMyAdmin
phpMyAdmin is a web interface through which you can manage your MySQL databases.
phpMyAdmin can be installed as follows:
yum install phpmyadmin
Now we configure phpMyAdmin. We change the Apache configuration so that phpMyAdmin allows
connections not just from localhost (by commenting out the
stanza):
vi /etc/httpd/conf.d/phpMyAdmin.conf
# phpMyAdmin - Web based MySQL browser written in php
#
# Allows only localhost by default
#
# But allowing phpMyAdmin to anyone other than localhost should be considered
# dangerous unless properly secured by SSL
Alias /phpMyAdmin /usr/share/phpMyAdmin
Alias /phpmyadmin /usr/share/phpMyAdmin
#
# order deny,allow
# deny from all
# allow from 127.0.0.1
# allow from ::1
#
# This directory does not require access over HTTP - taken from the original
# phpMyAdmin upstream tarball
#
Order Deny,Allow
Deny from All
Allow from None
# This configuration prevents mod_security at phpMyAdmin directories from
# filtering SQL etc. This may break your mod_security implementation.
#
#
#
# SecRuleInheritance Off
#
#
Restart Apache:
/etc/init.d/httpd restart
Afterwards, you can access phpMyAdmin under http://ec2-XXX-XX-XXX-XXX.compute-
1.amazonaws.com/phpmyadmin/:
Red Hat Enterprise Linux, CentOS, etc.
The OpenJDK 6 packages are also available in EPEL, a community-run project which makes Fedora
packages available to users of Red Hat Enterprise Linux 5, CentOS 5, and other RHEL 5 derivatives.
First install the package that enables the EPEL repository:
$ su -c "rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm"
If installing from a live image, such as a live CD or USB flash drive, add the OpenJDK 6 runtime after
the install:
* To install with the graphical package manager, follow Applications > Add/Remove Software and
use search to find the package(s) to install.
* To install from the command line:
$ su -c "yum install java-1.6.0-openjdk"
This package contains just the Java Runtime Environment. If you want to develop Java programs
then install the java-1.6.0-openjdk-devel package. You can also install all the OpenJDK 6 packages,
including the API documentation, by using the wildcard java-1.6.0-openjdk*.
On Fedora 8 the OpenJDK 7 runtime and development packages are installed by default during any
large-media install, such as from the Fedora 8 DVD, under the name IcedTea. If installing from a live
image then follow the above instructions but replace 1.6.0-openjdk with 1.7.0-icedtea.
yum install tomcat5 tomcat5-webapps tomcat5-admin-webapps
Set JAVA_HOME / PATH for single user
Login to your account and open .bash_profile file
$ vi ~/.bash_profile
Set JAVA_HOME as follows using syntax export JAVA_HOME=
to /usr/java/jdk1.5.0_07/bin/java, set it as follows:
export JAVA_HOME=/usr/java/jdk1.6.0_16/bin/java
Set PATH as follows:
export PATH=$PATH:/usr/java/jdk1.6.0_16/bin
Save and close the file. Just logout and login back to see new changes:
$ echo $JAVA_HOME
$ echo $PATH
Edit the .bash_profile for Java Environment
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_16/
export CATALINA_HOME=/opt/jetbrains/TeamCity
export PATH
export JAVA_HOME
unset USERNAME
http://www.jetbrains.com/teamcity/
Download TeamCity 5.1.2
http://download.jetbrains.com/teamcity/TeamCity-5.1.2.tar.gz
# cd /opt
# mkdir jetbrains
# cd jetbrains
# mkdir TeamCity
# cp TeamCity-5.1.2.tar.gz /opt/jetbrains/TeamCity
# vi /opt/jetnrains/TeamCity/conf/server.xml
connectionTimeout="20000"
redirectPort="8443"
enableLookup="false"
useBodyEncodingForURI="true"
/>
8543 will default port change to 8443
# service httpd restart
# chkconfig httpd on
# service mysqld restart
# chkconfig mysqld on
# service tomcat5 restart
# chkconfig tomcat5 on
# cd /opt/jetbrains/TeamCity/bin
TeamCity server can be started and stopped by the scripts provided in the
directory
To start/stop TeamCity server and default agent at the same time, use the runAll script.
To start/stop only the TeamCity server, use teamcity-server script.
For example:
* Use runAll.bat start to start the server and the default agent
* Use runAll.bat stop to stop the server and the default agent
By default, TeamCity runs on http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:8111/ and has
one registered build agent that runs on the same computer.
Tuesday, July 6, 2010
MySQL root user password using MySQL sql command
This is another method. MySQL stores username and passwords in user table inside MySQL database. You can directly update password using the following method to update or change password for user vivek:
1) Login to mysql server, type following command at shell prompt:
1) Login to mysql server, type following command at shell prompt:
$ mysql -u root -p2) Use mysql database (type command at mysql> prompt):
mysql> use mysql;3) Change password for user root:
mysql> update user set password=PASSWORD("NEWPASSWORD") where User='root';4) Reload privileges:
mysql> flush privileges; mysql> quit
Subscribe to:
Posts (Atom)