Friday, July 9, 2010

TeamCity with LAMP Cent OS AMIs

 Installing MySQL 5
To install MySQL, we do this:
yum install mysql mysql-server
Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the
system boots) and start the MySQL server:
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
Run
mysql_secure_installation
to set a password for the user root (otherwise anybody can access your MySQL database!):
[root@server1 ~]# mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
     SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none): <-- ENTER
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.
Set root password? [Y/n] <-- ENTER
New password: <-- yourrootsqlpassword
Re-enter new password: <-- yourrootsqlpassword
Password updated successfully!
Reloading privilege tables..
 ... Success!
By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] <-- ENTER
 ... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] <-- ENTER
 ... Success!
By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] <-- ENTER
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] <-- ENTER
 ... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MySQL
installation should now be secure.
Thanks for using MySQL!
Installing Apache2
Apache2 is available as a Fedora package, therefore we can install it like this:
yum install httpd
Now configure your system to start Apache at boot time...
chkconfig --levels 235 httpd on
... and start Apache:
/etc/init.d/httpd start
Now direct your browser to http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/ and you should see
the Apache2 placeholder
 page:
(( Optional ))
Installing PHP5
We can install PHP5 and the Apache PHP5 module as follows:
yum install php
We must restart Apache afterwards:
/etc/init.d/httpd restart
vi /var/www/html/info.php
phpinfo();
?>
Now we call that file in a browser (e.g. http://ec2-XX-xx-xxx-xxx.compute-
1.amazonaws.com/info.php):
Getting MySQL Support In PHP5
To get MySQL support in PHP, we can install the php-mysql package. It's a good idea to install some
other PHP5 modules as well as you might need them for your applications. You can search for available
PHP5 modules like this:
yum search php
Pick the ones you need and install them like this:
yum install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-
eaccelerator php-magickwand php-magpierss php-mbstring php-mcrypt php-mssql php-shout php-
snmp php-soap php-tidy
Now restart Apache2:
/etc/init.d/httpd restart
Now reload http://ec2-xXX-xX-xXx-XXx.compute-1.amazonaws.com/info.php in your browser and scroll
down to the modules section again. You should now find lots of new modules there, including the
MySQL module:
phpMyAdmin
phpMyAdmin is a web interface through which you can manage your MySQL databases.
phpMyAdmin can be installed as follows:
yum install phpmyadmin
Now we configure phpMyAdmin. We change the Apache configuration so that phpMyAdmin allows
connections not just from localhost (by commenting out the
stanza):
vi /etc/httpd/conf.d/phpMyAdmin.conf
# phpMyAdmin - Web based MySQL browser written in php
#
# Allows only localhost by default
#
# But allowing phpMyAdmin to anyone other than localhost should be considered
# dangerous unless properly secured by SSL
Alias /phpMyAdmin /usr/share/phpMyAdmin
Alias /phpmyadmin /usr/share/phpMyAdmin
#
# order deny,allow
# deny from all
# allow from 127.0.0.1
# allow from ::1
#

# This directory does not require access over HTTP - taken from the original
# phpMyAdmin upstream tarball
#

   Order Deny,Allow
   Deny from All
   Allow from None

# This configuration prevents mod_security at phpMyAdmin directories from
# filtering SQL etc. This may break your mod_security implementation.
#
#
#
#      SecRuleInheritance Off
#

#

Restart Apache:
/etc/init.d/httpd restart
Afterwards, you can access phpMyAdmin under http://ec2-XXX-XX-XXX-XXX.compute-
1.amazonaws.com/phpmyadmin/:
Red Hat Enterprise Linux, CentOS, etc.
The OpenJDK 6 packages are also available in EPEL, a community-run project which makes Fedora
packages available to users of Red Hat Enterprise Linux 5, CentOS 5, and other RHEL 5 derivatives.
First install the package that enables the EPEL repository:
   $ su -c "rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm"
If installing from a live image, such as a live CD or USB flash drive, add the OpenJDK 6 runtime after
the install:
   * To install with the graphical package manager, follow Applications > Add/Remove Software and
use search to find the package(s) to install.
   * To install from the command line:
       $ su -c "yum install java-1.6.0-openjdk"
     This package contains just the Java Runtime Environment. If you want to develop Java programs
then install the java-1.6.0-openjdk-devel package. You can also install all the OpenJDK 6 packages,
including the API documentation, by using the wildcard java-1.6.0-openjdk*.
On Fedora 8 the OpenJDK 7 runtime and development packages are installed by default during any
large-media install, such as from the Fedora 8 DVD, under the name IcedTea. If installing from a live
image then follow the above instructions but replace 1.6.0-openjdk with 1.7.0-icedtea.
 yum install tomcat5 tomcat5-webapps tomcat5-admin-webapps
Set JAVA_HOME / PATH for single user
Login to your account and open .bash_profile file
$ vi ~/.bash_profile
Set JAVA_HOME as follows using syntax export JAVA_HOME=. If your path is set
to /usr/java/jdk1.5.0_07/bin/java, set it as follows:
export JAVA_HOME=/usr/java/jdk1.6.0_16/bin/java
Set PATH as follows:
export PATH=$PATH:/usr/java/jdk1.6.0_16/bin
Save and close the file. Just logout and login back to see new changes:
$ echo $JAVA_HOME
$ echo $PATH
Edit the .bash_profile for Java Environment
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:
JAVA_HOME=/usr/java/jdk1.6.0_16/
export CATALINA_HOME=/opt/jetbrains/TeamCity
export PATH
export JAVA_HOME
unset USERNAME
http://www.jetbrains.com/teamcity/
Download TeamCity 5.1.2
http://download.jetbrains.com/teamcity/TeamCity-5.1.2.tar.gz
# cd /opt
# mkdir jetbrains
# cd jetbrains
# mkdir TeamCity
# cp TeamCity-5.1.2.tar.gz /opt/jetbrains/TeamCity
# vi /opt/jetnrains/TeamCity/conf/server.xml
 
           connectionTimeout="20000"
           redirectPort="8443"
           enableLookup="false"
           useBodyEncodingForURI="true"
      />
  
  
  
  
8543 will default port change to 8443
# service httpd restart
# chkconfig httpd on
# service mysqld restart
# chkconfig mysqld on
# service tomcat5 restart
# chkconfig tomcat5 on
# cd /opt/jetbrains/TeamCity/bin
TeamCity server can be started and stopped by the scripts provided in the /bin
directory
To start/stop TeamCity server and default agent at the same time, use the runAll script.
To start/stop only the TeamCity server, use teamcity-server script.
For example:
   * Use runAll.bat start to start the server and the default agent
   * Use runAll.bat stop to stop the server and the default agent
By default, TeamCity runs on http://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:8111/ and has
one registered build agent that runs on the same computer.

Tuesday, July 6, 2010

MySQL root user password using MySQL sql command

This is another method. MySQL stores username and passwords in user table inside MySQL database. You can directly update password using the following method to update or change password for user vivek:
1) Login to mysql server, type following command at shell prompt:
$ mysql -u root -p
2) Use mysql database (type command at mysql> prompt):
mysql> use mysql;
3) Change password for user root:
mysql> update user set password=PASSWORD("NEWPASSWORD") where User='root';
4) Reload privileges:
mysql> flush privileges;
mysql> quit

Wednesday, June 30, 2010

Linux Backup: Hard Disk Clone with "dd"

Hard Disk Clone

Suppose you have a 40GB hard disk and a removable hard disk whose capacity is 60GB, and you want to backup all the files from the hard disk to the removable disk. With "dd", it is a very easy task. Again, suppose your hard disk's Unix device name is /dev/sda and the removable disk is /dev/sdb. The following command can copy all the content from /dev/sda to /dev/sdb:
dd if=/dev/sda of=/dev/sdb
Here, if=... sets the source and of=... sets the destination. "dd" doesn't care of the contents of the hard disk. It just reads bytes from /dev/sda and writes them into /dev/sdb. It doesn't know what are files. So, the hard disk file system and how many partitions it has are not important. For example, if /dev/sda is splitted into three partitions, the /dev/sdb will have the same partitions. i.e. "destination" is completely same with "source".
Notice: to execute "dd" you should login as "root" or switch to "root" using "su" command. And you must be careful, a small mistake may cause a serious problem!

Making a Hard Disk Image File

Most of time you don't want to make a complete duplication of your hard disk. You may prefer to creating an image file of the hard disk and save it in other storage devices. The following command will create an image file "disk1.img" in your user's directory from /dev/sda:
dd if=/dev/sda of=~/disk1.img
Since you have created an image file, you can compress it with "gzip" or "bzip2":
gzip disk1.img #generates disk1.img.gz or
bzip2 disk1.img #generates disk1.img.bz2
You can save much storage space with compression. But it will take very long time.

Partition Clone

Backing up a hard disk partition is much similar to backing up a whole hard disk. The reason is that Unix/Linux uses device name, such as /dev/sda1, /dev/sda5... to indicate the partitions. For example, if you want to create an image file from the first partition of /dev/sda, use "dd" like this:
dd if=/dev/sda1 of=~/disk2.img
Also, you can compress the image file:
gzip disk2.img
By the way, you can copy a partition to another partition completely, just set "of" to the partition's device name. For example:
dd if=/dev/sda1 of=/dev/sdb5
This command will copy all the contents from /dev/sda1 to /dev/sdb5. You must be sure that the capacity of /dev/sdb5 is larger than /dev/sda1.

Restoring from an Image File

To restore a partition or a hard disk from an image file, just exchange the arguments "if" and "of". For example, restore the whole hard disk from the image file "disk1.img":
dd if=disk1.img of=/dev/sda
Restore the first partition of /dev/sda from the image file "disk2.img":
dd if=disk2.img of=/dev/sda1

Friday, June 25, 2010

Postfix Mail Server,amavisd-new, spam assassin, clamav and sqlgrey

Applicable to Fedora Versions

  • Fedora

Requirements

Mail servers rely on port 25 (tcp) to send and receive mail. It is also helpful to have a static IP address, however, it is not needed with todays offerings for dynamic DNS services. Some providers don't allow port 25 but this is mainly just restricted for "residential" lines. It is always good to keep Fedora updated and this howto assumes you are updated and running the latest versions for your release.
  • Port 25 inbound
  • Updated Fedora

Assumptions Made in HowTo

This howto assumes the following:
  • Domain name: example.com
  • Host name: host.example.com
  • Firewall is already configured to allow port 25:tcp
  • IPv4
  • Local user account: local-user

Doing the Work

The first steps are to just get everything installed that will be needed to complete the full howto. At this time this includes a few things that are not in Fedora Extras yet. This howto will be updated as software makes it into Fedora Extras. We will be using mysql server for postgrey. If you don't want postgrey or you want to use a different database backend, either don't install a database server or install your database server of choice. Information about using Postgre SQL will be added at some point.

  1. Install Needed Software
    1. Install most of the needed software from Fedora Extras using yum:
    2. yum install postfix mysql-server spamassassin clamav amavisd-new cyrus-sasl clamav-update sqlgrey

  2. Configure and Test Postfix
    1. Do some basic configuration to setup postfix before first starting it. Find the configuration variables and update them. Edit the /etc/postfix/main.cf configuration file and make the following changes:
    2. mydomain = example.com
      myorigin = $mydomain
      inet_interfaces = all
      mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
      mynetworks_style = host
    3. Start the server for the first time:
    4. /sbin/service postfix start
    5. Send a test mail to a local user using telnet:
    6. commands are in bold reponses are in italics
      telnet localhost 25
      Trying 127.0.0.1...
      Connected to localhost.localdomain (127.0.0.1).
      Escape character is '^]'.
      220 host.example.com ESMTP Postfix>
      EHLO testdomain.com
      250-host.example.com
      250-PIPELINING
      250-SIZE 10240000
      250-VRFY
      250-ETRN
      250-ENHANCEDSTATUSCODES
      250-8BITMIME
      MAIL FROM: 
      250 2.1.0 Ok
      
      RCPT TO: 
      250 2.1.5 Ok
      DATA
      354 End data with .
      Subject: Hello local-user
      Hey local-user,
      I just wanted to send some test mail to you :-)
      .250 2.0.0 Ok: queued as B95C8110064
      QUIT
    7. Check the users mail with the 'mail' command when logged in as the local-user:
    8. mail
      If this worked delete the users mail and move on, 'man mail' for more information about the mail command.

  3. Do Some More Configuration for Postfix
  4. This section start to configure postfix to be more secure. There are some options that are personal preferences of the author and may be changed. They are as follows:
    • Mail is stored in $HOME
    • TLS required for sending mail remotely
    • Certificates in /etc/postfix
    • The 'standard' is /etc/pki/tls
    1. Update the /etc/postfix/main.cf postfix configuration file and make the following changes:
      1. The following changes are updates:
      2. home_mailbox = Maildir/
      3. These changes are additions to the configuration file and may be added at the end of the file.
      4. #TLS - SMTP AUTH
        disable_vrfy_command = yes
        smtpd_use_tls = yes
        smtpd_tls_auth_only = yes
        tls_random_source = dev:/dev/urandom
        smtpd_tls_cert_file = /etc/postfix/cert.pem
        smtpd_tls_key_file = /etc/postfix/key.pem
        smtpd_sasl_auth_enable = yes
        smtpd_sasl_security_options = noanonymous
        broken_sasl_auth_clients = yes
        # Add some security
        smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination
    2. Move your certificates to the proper location (/etc/postfix/cert.pem and /etc/postfix/key.pem respectivly) and set proper permissions (600).
      If you don't have a certificate already, you may generate a self signed cert with the following commands:
      cd /etc/postfix
      openssl req -new -x509 -nodes -out cert.pem -keyout key.pem -days 3650
      chmod 600 *.pem
    3. Restart the server:
    4. /sbin/service postfix restart
    5. Try to send test mail to your local-user account both from localhost and a remote server.
    6. This should work. It would also be a good test to make sure that your server will not relay mail so try to send mail to another host using your server. It is recommended to continue to send testing mail with telnet so the maximum amount of information is available to debug what is going wrong. You should notice a new response from the server after you 'EHLO':
      EHLO testdomain.com
      250-host.example.com
      250-PIPELINING
      250-SIZE 10240000
      250-ETRN
      250-STARTTLS
      250-ENHANCEDSTATUSCODES
      250-8BITMIME
      250 DSN
    7. Test if TLS is working correctly:
    8. commands are in bold reponses are in italics
      telnet localhost 25
      Trying 127.0.0.1...
      Connected to localhost.localdomain (127.0.0.1).
      Escape character is '^]'.
      220 host.example.com ESMTP Postfix
      EHLO testdomain.com
      250-host.example.com
      250-PIPELINING
      250-SIZE 10240000
      250-ETRN
      250-STARTTLS
      250-ENHANCEDSTATUSCODES
      250-8BITMIME
      250 DSNSTARTTLS220 2.0.0 Ready to start TLS
      If you do not see 'Ready to start TLS', something is wrong with your TLS setup.

  5. Test SMTP Auth Using a Standard Mail Client
  6. Use your favorite mail client to test if SMTP auth is working. If TLS is not working, SMTP auth will also not work because this howto forces postfix to use TLS when doing SMTP auth.
    1. Start sasl:
    2. /sbin/service saslauthd start
    3. Start your favorite email client and send a test message to another server/mail system. Connect to the server with the following settings:
    4. In addition to a remote account, you could also send a test message to root, another account or yourself.
      • Host: host.example.com
      • User: local-user
      • Password: local-user's password
      • Force TLS for SMTP
      • Force SMTP Auth

  7. Setup Amavisd-New, Spam Assassin, Clam-AV
  8. Amavisd-new is the content filter that will run the spamassassin and clamav checks. It could also be configured to do other checks and has many other features. Those addtional features are outside the scope of this howto and might be added later.
    1. Configure amavisd-new. Make the following changes to the /etc/amavisd/amavisd.conf config file:
      $myhostname is only needed when the server has not been assigned a FQDN, however, it does not hurt to set the variable; check with the command 'hostname'
    2. $mydomain = 'example.com';
      $myhostname = 'host.example.com';
    3. Configure SpamAssassin to do extended checks such as rbl, pyzor, razor2, etc. Make the following changes to the /etc/mail/spamassassin/local.cf config file:
    4. report_safe             1
      use_bayes               1
      bayes_auto_learn        1
      skip_rbl_checks         0
      use_razor2              1
      use_dcc                 1
      use_pyzor               1
      whitelist_from *@example.com
    5. Enable ClamAV to do automatic updates to virus definitions. Make the following changes to /etc/sysconfig/freshclam:
      Note: The change is to comment out this line.
    6. #FRESHCLAM_DELAY=disabled-warn  # REMOVE ME
    7. Update /etc/freshclam.conf to enable automatic updates:
      Note: The change is to comment out 'Example'.
      #Example
    8. Start everything up:
    9. /sbin/service amavisd start
      /sbin/service clamd.amavisd start
      /sbin/service spamassassin start

  9. Configure Postfix to Use the New Content Filtering System
  10. Postfix needs to be told to use the new content filtering system. A few things need to be changed to enable the new filtering system.
    1. Add the following to /etc/postfix/master.cf:
    2. smtp-amavis unix -      -       n       -       2       smtp
          -o smtp_data_done_timeout=1200
          -o smtp_send_xforward_command=yes
          -o disable_dns_lookups=yes
          -o max_use=20
      
      
      127.0.0.1:10025 inet n  -       n       -       -  smtpd
          -o content_filter=
          -o local_recipient_maps=
          -o relay_recipient_maps=
          -o smtpd_restriction_classes=
          -o smtpd_delay_reject=no
          -o smtpd_client_restrictions=permit_mynetworks,reject
          -o smtpd_helo_restrictions=
          -o smtpd_sender_restrictions=
          -o smtpd_recipient_restrictions=permit_mynetworks,reject
          -o smtpd_data_restrictions=reject_unauth_pipelining
          -o smtpd_end_of_data_restrictions=
          -o mynetworks=127.0.0.0/8
          -o smtpd_error_sleep_time=0
          -o smtpd_soft_error_limit=1001
          -o smtpd_hard_error_limit=1000
          -o smtpd_client_connection_count_limit=0
          -o smtpd_client_connection_rate_limit=0
          -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
    3. Add the following to the /etc/postfix/main.cf config file:
    4. content_filter = smtp-amavis:[127.0.0.1]:10024
    5. Restart postfix to apply the changes:
    6. /sbin/service postfix restart

  11. Setup Grey Listing
  12. Grey listing is an anti-spam technique that is used to twart spammers from doing drive by spamming. There are two steps to get it working with postfix. Setting up the mysql database and then enabiling the checks. You may use any supported database you would like but additional database configurations are outside of the scope of this howto. Replace sensitive information such as passwords with unique settings.
    1. Setup the mysql database:
    2. This assumes mysql server has not been setup and we are dealing with a fresh configuration. If mysql is already setup, you will need to use the '-p' switch for the mysql commands and there is no reason to set a new mysql root password. Also note, you may use whatever user/database name you want but this will need to be updated in the conf file.
      /sbin/service mysqld start
      mysql -u root
    3. This will bring you to the mysql shell where you can add the needed user and database for sqlgrey:
    4. Commands are in bold responses are in italics
      mysql> create database sqlgrey;
      Query OK, 1 row affected (0.01 sec)
      mysql> grant all on sqlgrey.* to sqlgrey@localhost identified by 'mysqlUserPassword';
      Query OK, 0 rows affected (0.01 sec)
      mysql> quit
      Bye
    5. Set a root password for mysql:
    6. mysqladmin -u root password "mysqlRootPassword"
    7. Configure sqlgrey for the database. Make the following changes to the /etc/sqlgrey/sqlgrey.conf config:
    8. db_type = mysql
      db_pass = mysqlUserPassword
      admin_mail = server-admin@example.com
    9. Start the sqlgrey service:
    10. /sbin/service sqlgrey start

  13. Setup Postfix to Do Grey Listing
  14. Postfix needs to be configured to check the greylisting service for the status of a sender.
    1. Configure postfix to do the greylist check. Make the following update to the /etc/postfix/main.cf config file:
    2. smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, check_policy_service inet:127.0.0.1:2501
    3. Restart postfix to apply the changes:
    4. /sbin/service postfix restart

  15. Set Services to Run on Boot
  16. The combination of services need to get set to run on boot. Do so with 'chkconfig':
    /sbin/chkconfig postfix on
    /sbin/chkconfig amavisd on
    /sbin/chkconfig clamd.amavisd on
    /sbin/chkconfig spamassassin on
    /sbin/chkconfig mysqld on
    /sbin/chkconfig sqlgrey on