Linux Backup: Hard Disk Clone with "dd"

Hard Disk Clone

Suppose you have a 40GB hard disk and a removable hard disk whose capacity is 60GB, and you want to backup all the files from the hard disk to the removable disk. With "dd", it is a very easy task. Again, suppose your hard disk's Unix device name is /dev/sda and the removable disk is /dev/sdb. The following command can copy all the content from /dev/sda to /dev/sdb:
dd if=/dev/sda of=/dev/sdb
Here, if=... sets the source and of=... sets the destination. "dd" doesn't care of the contents of the hard disk. It just reads bytes from /dev/sda and writes them into /dev/sdb. It doesn't know what are files. So, the hard disk file system and how many partitions it has are not important. For example, if /dev/sda is splitted into three partitions, the /dev/sdb will have the same partitions. i.e. "destination" is completely same with "source".
Notice: to execute "dd" you should login as "root" or switch to "root" using "su" command. And you must be careful, a small mistake may cause a serious problem!

Making a Hard Disk Image File

Most of time you don't want to make a complete duplication of your hard disk. You may prefer to creating an image file of the hard disk and save it in other storage devices. The following command will create an image file "disk1.img" in your user's directory from /dev/sda:
dd if=/dev/sda of=~/disk1.img
Since you have created an image file, you can compress it with "gzip" or "bzip2":
gzip disk1.img #generates disk1.img.gz or
bzip2 disk1.img #generates disk1.img.bz2
You can save much storage space with compression. But it will take very long time.

Partition Clone

Backing up a hard disk partition is much similar to backing up a whole hard disk. The reason is that Unix/Linux uses device name, such as /dev/sda1, /dev/sda5... to indicate the partitions. For example, if you want to create an image file from the first partition of /dev/sda, use "dd" like this:
dd if=/dev/sda1 of=~/disk2.img
Also, you can compress the image file:
gzip disk2.img
By the way, you can copy a partition to another partition completely, just set "of" to the partition's device name. For example:
dd if=/dev/sda1 of=/dev/sdb5
This command will copy all the contents from /dev/sda1 to /dev/sdb5. You must be sure that the capacity of /dev/sdb5 is larger than /dev/sda1.

Restoring from an Image File

To restore a partition or a hard disk from an image file, just exchange the arguments "if" and "of". For example, restore the whole hard disk from the image file "disk1.img":
dd if=disk1.img of=/dev/sda
Restore the first partition of /dev/sda from the image file "disk2.img":
dd if=disk2.img of=/dev/sda1

Postfix Mail Server,amavisd-new, spam assassin, clamav and sqlgrey

Applicable to Fedora Versions

• Fedora

Requirements

Mail servers rely on port 25 (tcp) to send and receive mail. It is also helpful to have a static IP address, however, it is not needed with todays offerings for dynamic DNS services. Some providers don't allow port 25 but this is mainly just restricted for "residential" lines. It is always good to keep Fedora updated and this howto assumes you are updated and running the latest versions for your release.

• Port 25 inbound
• Updated Fedora

Assumptions Made in HowTo

This howto assumes the following:

• Domain name: example.com
• Host name: host.example.com
• Firewall is already configured to allow port 25:tcp
• IPv4
• Local user account: local-user

Doing the Work

The first steps are to just get everything installed that will be needed to complete the full howto. At this time this includes a few things that are not in Fedora Extras yet. This howto will be updated as software makes it into Fedora Extras. We will be using mysql server for postgrey. If you don't want postgrey or you want to use a different database backend, either don't install a database server or install your database server of choice. Information about using Postgre SQL will be added at some point.

Install Needed Software

1. Install most of the needed software from Fedora Extras using yum:

yum install postfix mysql-server spamassassin clamav amavisd-new cyrus-sasl clamav-update sqlgrey

Configure and Test Postfix

1. Do some basic configuration to setup postfix before first starting it. Find the configuration variables and update them. Edit the /etc/postfix/main.cf configuration file and make the following changes:

mydomain = example.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
mynetworks_style = host

2. Start the server for the first time:

/sbin/service postfix start

3. Send a test mail to a local user using telnet:
commands are in bold reponses are in italics

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 host.example.com ESMTP Postfix>
EHLO testdomain.com
250-host.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
MAIL FROM: 
250 2.1.0 Ok

RCPT TO: 
250 2.1.5 Ok
DATA
354 End data with .
Subject: Hello local-user
Hey local-user,
I just wanted to send some test mail to you :-)
.250 2.0.0 Ok: queued as B95C8110064
QUIT

4. Check the users mail with the 'mail' command when logged in as the local-user:

mail

If this worked delete the users mail and move on, 'man mail' for more information about the mail command.

Do Some More Configuration for Postfix

This section start to configure postfix to be more secure. There are some options that are personal preferences of the author and may be changed. They are as follows:

• Mail is stored in $HOME
• TLS required for sending mail remotely
• Certificates in /etc/postfix
• The 'standard' is /etc/pki/tls

1. Update the /etc/postfix/main.cf postfix configuration file and make the following changes:
1. The following changes are updates:

home_mailbox = Maildir/

2. These changes are additions to the configuration file and may be added at the end of the file.

#TLS - SMTP AUTH
disable_vrfy_command = yes
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
tls_random_source = dev:/dev/urandom
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_key_file = /etc/postfix/key.pem
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
# Add some security
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination

2. Move your certificates to the proper location (/etc/postfix/cert.pem and /etc/postfix/key.pem respectivly) and set proper permissions (600).
   
   If you don't have a certificate already, you may generate a self signed cert with the following commands:

   cd /etc/postfix
   openssl req -new -x509 -nodes -out cert.pem -keyout key.pem -days 3650
   chmod 600 *.pem

3. Restart the server:

/sbin/service postfix restart

4. Try to send test mail to your local-user account both from localhost and a remote server.
This should work. It would also be a good test to make sure that your server will not relay mail so try to send mail to another host using your server. It is recommended to continue to send testing mail with telnet so the maximum amount of information is available to debug what is going wrong. You should notice a new response from the server after you 'EHLO':

EHLO testdomain.com
250-host.example.com
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

5. Test if TLS is working correctly:
commands are in bold reponses are in italics

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 host.example.com ESMTP Postfix
EHLO testdomain.com
250-host.example.com
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSNSTARTTLS220 2.0.0 Ready to start TLS

If you do not see 'Ready to start TLS', something is wrong with your TLS setup.

Test SMTP Auth Using a Standard Mail Client

Use your favorite mail client to test if SMTP auth is working. If TLS is not working, SMTP auth will also not work because this howto forces postfix to use TLS when doing SMTP auth.
1. Start sasl:

/sbin/service saslauthd start

2. Start your favorite email client and send a test message to another server/mail system. Connect to the server with the following settings:
In addition to a remote account, you could also send a test message to root, another account or yourself.
• Host: host.example.com
• User: local-user
• Password: local-user's password
• Force TLS for SMTP
• Force SMTP Auth

Setup Amavisd-New, Spam Assassin, Clam-AV

Amavisd-new is the content filter that will run the spamassassin and clamav checks. It could also be configured to do other checks and has many other features. Those addtional features are outside the scope of this howto and might be added later.
1. Configure amavisd-new. Make the following changes to the /etc/amavisd/amavisd.conf config file:
   
   $myhostname is only needed when the server has not been assigned a FQDN, however, it does not hurt to set the variable; check with the command 'hostname'

$mydomain = 'example.com';
$myhostname = 'host.example.com';

2. Configure SpamAssassin to do extended checks such as rbl, pyzor, razor2, etc. Make the following changes to the /etc/mail/spamassassin/local.cf config file:

report_safe             1
use_bayes               1
bayes_auto_learn        1
skip_rbl_checks         0
use_razor2              1
use_dcc                 1
use_pyzor               1
whitelist_from *@example.com

3. Enable ClamAV to do automatic updates to virus definitions. Make the following changes to /etc/sysconfig/freshclam:
   
   Note: The change is to comment out this line.

#FRESHCLAM_DELAY=disabled-warn  # REMOVE ME

4. Update /etc/freshclam.conf to enable automatic updates:
   
   Note: The change is to comment out 'Example'.

   #Example

5. Start everything up:

/sbin/service amavisd start
/sbin/service clamd.amavisd start
/sbin/service spamassassin start

Configure Postfix to Use the New Content Filtering System

Postfix needs to be told to use the new content filtering system. A few things need to be changed to enable the new filtering system.
1. Add the following to /etc/postfix/master.cf:

smtp-amavis unix -      -       n       -       2       smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20


127.0.0.1:10025 inet n  -       n       -       -  smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

2. Add the following to the /etc/postfix/main.cf config file:

content_filter = smtp-amavis:[127.0.0.1]:10024

3. Restart postfix to apply the changes:

/sbin/service postfix restart

Setup Grey Listing

Grey listing is an anti-spam technique that is used to twart spammers from doing drive by spamming. There are two steps to get it working with postfix. Setting up the mysql database and then enabiling the checks. You may use any supported database you would like but additional database configurations are outside of the scope of this howto. Replace sensitive information such as passwords with unique settings.
1. Setup the mysql database:
This assumes mysql server has not been setup and we are dealing with a fresh configuration. If mysql is already setup, you will need to use the '-p' switch for the mysql commands and there is no reason to set a new mysql root password. Also note, you may use whatever user/database name you want but this will need to be updated in the conf file.

/sbin/service mysqld start
mysql -u root

2. This will bring you to the mysql shell where you can add the needed user and database for sqlgrey:
Commands are in bold responses are in italics

mysql> create database sqlgrey;
Query OK, 1 row affected (0.01 sec)
mysql> grant all on sqlgrey.* to sqlgrey@localhost identified by 'mysqlUserPassword';
Query OK, 0 rows affected (0.01 sec)
mysql> quit
Bye

3. Set a root password for mysql:

mysqladmin -u root password "mysqlRootPassword"

4. Configure sqlgrey for the database. Make the following changes to the /etc/sqlgrey/sqlgrey.conf config:

db_type = mysql
db_pass = mysqlUserPassword
admin_mail = server-admin@example.com

5. Start the sqlgrey service:

/sbin/service sqlgrey start

Setup Postfix to Do Grey Listing

Postfix needs to be configured to check the greylisting service for the status of a sender.
1. Configure postfix to do the greylist check. Make the following update to the /etc/postfix/main.cf config file:

smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, check_policy_service inet:127.0.0.1:2501

2. Restart postfix to apply the changes:

/sbin/service postfix restart

Set Services to Run on Boot

The combination of services need to get set to run on boot. Do so with 'chkconfig':

/sbin/chkconfig postfix on
/sbin/chkconfig amavisd on
/sbin/chkconfig clamd.amavisd on
/sbin/chkconfig spamassassin on
/sbin/chkconfig mysqld on
/sbin/chkconfig sqlgrey on

Ubuntu vsftpd ftp service / server install, configuration

Ubuntu Linux comes with various ftp servers to setup FTP service such as:
=> proftpd - Versatile, virtual-hosting FTP daemon
=> vsftpd - The Very Secure FTP Daemon
=> ftpd - FTP server
=> wu-ftpd - powerful and widely used FTP server
=> wzdftpd - A portable, modular, small and efficient ftp server
=> pure-ftpd - Pure-FTPd FTP server
I recommend using vsftpd. It is simple and quite secure FTP server. According to vsftpd man page:

vsftpd is the Very Secure File Transfer Protocol Daemon. The server can be launched via a super-server such as inetd or xinetd. Alternatively, vsftpd can be launched in standalone mode, in which case vsftpd itself will listen on the network.

=> Default ftp port : 21
=> Default configuration file : /etc/vsftpd.conf

How do I set up the vsftpd daemon to accept connections from another computer?

The configuration of the vsftpd FTP service (read as daemon ) simply requires three steps.

Step # 1: Install vsftpd

Type apt-get command to install vsftpd
$ sudo apt-get install vsftpd
Output:

Password:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  vsftpd
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 121kB of archives.
After unpacking 438kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com edgy/main vsftpd 2.0.4-0ubuntu5 [121kB]
Fetched 121kB in 0s (246kB/s)
Selecting previously deselected package vsftpd.
(Reading database ... 31396 files and directories currently installed.)
Unpacking vsftpd (from .../vsftpd_2.0.4-0ubuntu5_amd64.deb) ...
Setting up vsftpd (2.0.4-0ubuntu5) ...
Adding system user `ftp' with uid 106...
Adding new user `ftp' (106) with group `nogroup'.
Not creating home directory `/home/ftp'.
 * Starting FTP server: vsftpd

Step # 2: Configure /etc/vsftpd.conf

The default vsftpd configuration file is /etc/vsftpd.conf. You need to edit this file using text editor such as vi:
$ sudo vi /etc/vsftpd.conf
Add the following line (uncomment line) to the vsftpd configuration file:
local_enable=YES
Above config directive will allow local users to log in via ftp
If you would like to allow users to upload file, add the following to the file:
write_enable=YES
For security you may restrict local users to their home directories. Add the following to the file:
chroot_local_user=YES
Save and close the file.

Step # 3: Restart vsftpd

To restart vsftpd type the command :
$ sudo /etc/init.d/vsftpd restart
Output:

* Stopping FTP server: vsftpd                                                                                       [ ok ]
 * Starting FTP server: vsftpd                                                                                       [ ok ]

How do I use ftp command line utility?

Now you should be able to FTP to this server with any account that exists on the system except for the root user. From Windows or other Linux system use ftp client, type the command:
$ ftp ftp.ossc.in
Output:

Connected to ftp.ossc.in.
220 (vsFTPd 2.0.4)
Name (ftp.ossc.in:rajat): rajat
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> pwd
257 "/"
ftp> ls
200 PORT command successful. Consider using PASV.
150 Here comes the directory listing.
-rw-r--r--    1 1000     1000        91798 Aug 16 08:26 apf-current.tar.gz
-rwxr-xr-x    1 1000     1000          156 Nov 10 07:05 iptables.stop
drwxr-xr-x    3 0        0            4096 Dec 23 11:11 postfix
-rw-r--r--    1 0        0        10481942 Nov 29 23:35 webmin_1.310_all.deb
226 Directory send OK.
ftp> quit
221 Goodbye.

Open FTP port using iptables (optional)

Add following rules to your iptables script. Assuming that default incoming policy is drop. If protocol is TCP and destination port is 21 (ftp):
iptables -A INPUT -p tcp -m tcp --dport 21 -j ACCEPT
See - How do I open open ftp port 21 using iptables for more information.
There are a large number of other configuration options available for vsftpd that can be used to fine tune ftp server. Read vsftpd.conf man page by typing following command:
$ man vsftpd.conf

LAMP on Drupal Installation and configuration

Drupal Install and configuration on LAMP
yum install mysql mysql-server

Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the system boots) and start the MySQL server:
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start

[root@rajat Rajat]# service mysqld restart
Stopping mysqld: [ OK ]
Initializing MySQL database: Installing MySQL system tables...
OK
Filling help tables...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h rajat.synovaindia.com password 'new-password'

Alternatively you can run:
/usr/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /usr/mysql-test ; perl mysql-test-run.pl

Please report any problems with the /usr/bin/mysqlbug script!

[ OK ]
Starting mysqld: [ OK ]
[root@rajat Rajat]# /usr/bin/mysqladmin -u root password 'password'

Apache2 is available as a Fedora package, therefore we can install it like this:
yum install httpd
Now configure your system to start Apache at boot time...
chkconfig --levels 235 httpd on
... and start Apache:
/etc/init.d/httpd start
[root@rajat default]# service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]

We can install PHP5 and the Apache PHP5 module as follows:
yum install php
We must restart Apache afterwards:
/etc/init.d/httpd restart
Testing PHP5 / Getting Details About Your PHP5 Installation
vi /var/www/html/info.php

http://rajat.synovaindia.com/info.php

yum install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-
eaccelerator php-magickwand php-magpierss php-mapserver php-mbstring php-mcrypt php-mssql php-
shout php-snmp php-soap php-tidy phpmyadmin
[root@rajat default]# service httpd restart
Stopping httpd:                              [ OK ]
Starting httpd:                             [ OK ]

http://drupal.org/project/drupal

tar -xvzf drupal-6.17.tar.gz /var/www/html/


goto
/var/www/html/sites/default
cp default.settings.php setting.php
chmod 777 settings.php

#chmod 755 settings.php

Install and configure munin for Server Monitoring

Munin is an efficient and lightweight Monitoring tool. Its available in ’server’ and ‘node’ packages. You can install server in a single machine and node in all other machines and then view all information/statistical data in server machine.
This quick copy and paste howto explains steps needed to setup server and multiple nodes. These steps are tested in CentOS 5.X Server.

Step 1. Prepare Server machine. For quick installation and dependency resolution, you can install DAG repository:
# rpm -Uhv http://apt.sw.be/packages/rpmforge-release/rpmforge-release-0.3.6-1.elX.rf.i386.rpm
Step 2. Install munin (server) and munin-node (node) on this machine:
# yum install munin munin-node
Step 3. Setup a directory to store munin generated htmls, you can create a directory in your server DocumentRoot or in munin directory itself. Here we create it in munin’s directory:
# cd /etc/munin
# mkdir html
# chown munin:munin html -R
Step 4. If in previous step, you created a directory in your web server root, then you dont need to set an alias, else if you did the exact whats written in above step, create an alias:
# vi /etc/httpd/conf/httpd.conf
—–
Alias /server-health/ “/etc/munin/html/”

AllowOverride None
Options None
Order allow,deny
Allow from all

—–
Step 5. Configure munin for monitoring localhost (you are already in /etc/munin directory):
# vi munin.conf
—-
htmldir /etc/munin/html
[localhost]
address 127.0.0.1
use_node_name yes
—-
here we set the html directory and section for local machine. Feel free to change ‘localhost’ to any meaningful name like Monitoring Server etc. You can further read the file for more options.
Step 6. Start munin service:
# chkconfig munin-node on
# service munin-node start
Step 7. Restart your web server and then access the munin page from your browser by pointing it to http://localhost/server-health/. You can also password protect the munin page by using .htaccess or inserting rules in your httpd.conf file but its optional.
You should be able to view various parameters/values/trends related to disk,network,apache,mail etc of your local machine/server via browser. Let us configure multiple nodes now:
Step 8. SSH to any other machine/ server and install DAG repository as described in Step 1 and then install the munin node:
# yum install munin-node
Step 10. Configure the node to allow our main server get input from this machine. For this purpose, we need to add ‘Allow IP ADDRESS’ line in config file. Note that there’s already a Allow line which allow localhost to get values, we just need to add another line for server:
# vi /etc/munin/munin-node.conf
—
allow ^192\.168\.0\.1$
—
Here 192.168.0.1 (change it as per your settings) is the IP of server machine, written in regex form which is required here.
Step 11. Start the node now:
# chkconfig munin-node on
# service munin-node start
Now configure the Server to monitor the above machine:
Step 12. Goto Server machine and update munin.conf and add a new section:
# vi /etc/munin.conf
—
[App_Server]
address 192.168.0.5
use_node_name yes
—
Here, App_Server is the name (you can write any), and 192.168.0.5 is the IP address of the machine in which we configured munin-node in Step 9 to Step 11.
After a while, Check you browser and munin should include the update from node machine.

Chmod Command

Following are the symbolic representation of three different roles:

■u is for user,
■g is for group,
■and o is for others.
Following are the symbolic representation of three different permissions:

■r is for read permission,
■w is for write permission,
■x is for execute permission.
Following are few examples on how to use the symbolic representation on chmod.

1. Add single permission to a file/directory
Changing permission to a single set. + symbol means adding permission. For example, do the following to give execute permission for the user irrespective of anything else:

$ chmod u+x filename2. Add multiple permission to a file/directory
Use comma to separate the multiple permission sets as shown below.

$ chmod u+r,g+x filename3. Remove permission from a file/directory
Following example removes read and write permission for the user.

$ chmod u-rx filename4. Change permission for all roles on a file/directory
Following example assigns execute privilege to user, group and others (basically anybody can execute this file).

$ chmod a+x filename5. Make permission for a file same as another file (using reference)
If you want to change a file permission same as another file, use the reference option as shown below. In this example, file2’s permission will be set exactly same as file1’s permission.

$ chmod --reference=file1 file26. Apply the permission to all the files under a directory recursively
Use option -R to change the permission recursively as shown below.

$ chmod -R 755 directory-name/7. Change execute permission only on the directories (files are not affected)
On a particular directory if you have multiple sub-directories and files, the following command will assign execute permission only to all the sub-directories in the current directory (not the files in the current directory).

$ chmod u+X *