Install Remastersys in Ubuntu (http://sourceforge.net/projects/remastersys/files/ download lates version for Ubuntu 10.04)
The Remastersys repository needs to be added to your /etc/apt/sources.list
sudo vi /etc/apt/sources.list
Paste the following into the sources.list:
# Remastersys
deb http://www.remastersys.klikit-linux.com/repository remastersys/
Save and exit the file.
Update the source list using the following command
sudo apt-get update
Install remastersys using the following command
sudo apt-get install remastersys
This will complete the installation
Using Remastersys
In order to learn how you can use remastersys, run
sudo remastersys
remastersys Syntax
sudo remastersys backup|clean|dist [cdfs|iso] [filename.iso]
remastersys Examples
1) to make a livecd/dvd backup of your system
sudo remastersys backup
2) to make a livecd/dvd backup and call the iso custom.iso
sudo remastersys backup custom.iso
3) to clean up temporary files of remastersys
sudo remastersys clean
4) to make a distributable livecd/dvd of your system
sudo remastersys dist
5) to make a distributable livecd/dvd filesystem only
sudo remastersys dist cdfs
6) to make a distributable iso named custom.iso but only if the cdfs is already present
sudo remastersys dist iso custom.iso
cdfs and iso options should only be used if you wish to modify something on the cd before the iso is created. An example of this would be to modify the isolinux portion of the livecd/dvd
Creating An ISO Image
To create an iso image of your installation, simply run
sudo remastersys dist
This will create an iso image called customdist.iso in the /home/remastersys directory. The dist option makes that your personal folder (e.g. /home/rajat) will not be included in the iso image. You might have to insert your Ubuntu installation CD during the process.
This is how the end of the process looks:
[...]
92.16% done, estimate finish Tue MAY 11 13:31:20 2010
93.39% done, estimate finish Tue MAY 11 13:31:21 2010
94.62% done, estimate finish Tue MAY 11 13:31:22 2010
95.85% done, estimate finish Tue MAY 11 13:31:23 2010
97.08% done, estimate finish Tue MAY 11 13:31:24 2010
98.31% done, estimate finish Tue MAY 11 13:31:25 2010
99.54% done, estimate finish Tue MAY 11 13:31:26 2010
Total translation table size: 2048
Total rockridge attributes bytes: 3950
Total directory bytes: 9094
Path table size(bytes): 54
Max brk space used 0
406890 extents written (794 MB)
/home/remastersys/customdist.iso is ready to be burned or tested in a virtual machine.
Check the size and if it is larger than 700MB you will need to burn it to a dvd
796M /home/remastersys/customdist.iso
Clean Up
After you’ve burnt the iso image onto a CD/DVD, you can run
sudo remastersys clean
to remove all temporary file created during the iso generation as well as the /home/remastersys directory.
Monday, May 10, 2010
Wednesday, May 5, 2010
Fedora NFS Server Configuration
1. Download and install NFS rpm package using yum
# yum -y install nfs-utils rpcbind
If you have an earlier Fedora versions
# yum -y intall nfs-utils portmap
2. Prepare and modify /etc/exports for sharing files, that would similar to as shown below
/home/NFS-files 192.168.100.0/24(ro,sync)
/home/NFS-share */26(rw,sync)
/ISO 192.168.100.0/24(ro,sync)
Save and exit.
Legend:
rw = for read/write access
ro = for read only access
* = allow any source request
/24 , /26 = IP network subnet
Create NFS folder and populate your NFS files you wish to share
# mkdir /home/NFS-files /home/NFS-share /ISO
# cp your-files /home/NFS-files
3. Start your NFS service
# service nfs start
# service rpcbind start
For earlier Fedora versions
# service portmap start
# service nfs start
If you wish to modify any other non-default NFS diretives, modify /etc/sysconfig/nfs .
To restart NFS service
# service nfs restart
# service rpcbind restart
For earlier Fedora versions, instead of rpcbind,
# service portmap restart
To monitor NFS stats
The nfsstat command displays statistical information about RPC calls.
# nfsstats
# nfsstats -o net
Further NFSman page
# man exports
# yum -y install nfs-utils rpcbind
If you have an earlier Fedora versions
# yum -y intall nfs-utils portmap
2. Prepare and modify /etc/exports for sharing files, that would similar to as shown below
/home/NFS-files 192.168.100.0/24(ro,sync)
/home/NFS-share */26(rw,sync)
/ISO 192.168.100.0/24(ro,sync)
Save and exit.
Legend:
rw = for read/write access
ro = for read only access
* = allow any source request
/24 , /26 = IP network subnet
Create NFS folder and populate your NFS files you wish to share
# mkdir /home/NFS-files /home/NFS-share /ISO
# cp your-files /home/NFS-files
3. Start your NFS service
# service nfs start
# service rpcbind start
For earlier Fedora versions
# service portmap start
# service nfs start
If you wish to modify any other non-default NFS diretives, modify /etc/sysconfig/nfs .
To restart NFS service
# service nfs restart
# service rpcbind restart
For earlier Fedora versions, instead of rpcbind,
# service portmap restart
To monitor NFS stats
The nfsstat command displays statistical information about RPC calls.
# nfsstats
# nfsstats -o net
Further NFSman page
# man exports
Thursday, April 29, 2010
Mysql with RHEL/Centos/Fedora
1. Identify the Group name of MySQL Packages
yum grouplist displays all package groups that are available in the repository. As shown below, mysql package group is called “MySQL Database”.
# yum grouplist | grep -i mysql
MySQL Database
2. What is bundled in the “MySQL Database” group?
yum groupinfo displays all the packages that are bundled in a group. This displays the mandatory, default and optional packages that are available in that particular group.
As shown below, “MySQL Database” group contains 1 mandatory package, 6 default packages, and 5 optional packages.
# yum groupinfo "MySQL Database"
Group: MySQL Database
Description: This package group contains packages useful for use with MySQL.
Mandatory Packages:
mysql
Default Packages:
MySQL-python
libdbi-dbd-mysql
mysql-connector-odbc
mysql-server
perl-DBD-MySQL
unixODBC
Optional Packages:
mod_auth_mysql
mysql-bench
mysql-devel
php-mysql
qt-MySQL
3. Install the “MySQL Database” group using yum groupinstall
yum groupinstall will install the “MySQL Database” group of packages as shown below.
# yum groupinstall "MySQL Database"
Resolving Dependencies
Dependencies Resolved
Transaction Summary
=========================
Install 12 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Installed:
MySQL-python.i386 0:1.2.1-1 libdbi-dbd-mysql.i386 0:0.8.1a-1.2.2
mysql.i386 0:5.0.77-4.el5_4.2 mysql-connector-odbc.i386 0:3.51.26r1127-1.el5
mysql-server.i386 0:5.0.77-4.el5_4.2 perl-DBD-MySQL.i386 0:3.0007-2.el5
unixODBC.i386 0:2.2.11-7.1
Dependency Installed:
libdbi.i386 0:0.8.1-2.1 libdbi-drivers.i386 0:0.8.1a-1.2.2
libtool-ltdl.i386 0:1.5.22-7.el5_4
mx.i386 0:2.0.6-2.2.2 perl-DBI.i386 0:1.52-2.el5
Complete!
Note: If you are having some issues during the installation, verify the full mysql install log to see what you are missing.
4. Verify MySQL Installation
Execute rpm -qa, to confirm that the mysql related packages are installed.
# rpm -qa | grep -i mysql
MySQL-python-1.2.1-1
mysql-5.0.77-4.el5_4.2
mysql-connector-odbc-3.51.26r1127-1.el5
mysql-server-5.0.77-4.el5_4.2
libdbi-dbd-mysql-0.8.1a-1.2.2
perl-DBD-MySQL-3.0007-2.el5
Check the /etc/passwd and /etc/group to make sure it has created a mysql username and group.
# grep mysql /etc/passwd
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
# grep mysql /etc/group
mysql:x:27:
5. MySQL Post installation – Execute mysql_install_db
mysql_install_db program will setup the necessary grant tables. The mysql_install_db program gets executed as part of the rpm installation. But, it doesn’t hurt to execute the mysql_install_db program again to make sure the grant tables are setup properly.
# /usr/bin/mysql_install_db --user=mysql
Installing MySQL system tables...OK
Filling help tables...OK
.....
The latest information about MySQL is available on the web at http://www.mysql.com
6. Start MySQL Server
# service mysqld status
mysqld is stopped
# service mysqld start
Starting MySQL: [ OK ]
7. Verify that the MySQL server is up and running.
# /usr/bin/mysqladmin version
/usr/bin/mysqladmin Ver 8.41 Distrib 5.0.77, for redhat-linux-gnu on i686
Copyright (C) 2000-2006 MySQL AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license
Server version 5.0.77
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 39 sec
Threads: 1 Questions: 2 Slow queries: 0 Opens: 12 Flush tables: 1
Open tables: 6 Queries per second avg: 0.051
# /usr/bin/mysqlshow
+--------------------+
| Databases |
+--------------------+
| information_schema |
| mysql |
| test |
+--------------------+
# /usr/bin/mysqlshow mysql
Database: mysql
+---------------------------+
| Tables |
+---------------------------+
| columns_priv |
| db |
| func |
| help_category |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
Stop and start the mysql server again to make sure they are no issues.
# service mysqld stop
Stopping MySQL: [ OK ]
# service mysqld start
Starting MySQL: [ OK ]
8. Change the MySQL root account password
Change the MySQL root account password to something secure.
# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.0.77 Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> select host, user from mysql.user;
+-----------+------+
| host | user |
+-----------+------+
| 127.0.0.1 | root |
| localhost | |
| localhost | root |
+-----------+------+
5 rows in set (0.00 sec)
mysql> set password for 'root'@'localhost' = PASSWORD('DoNotTell$AnyBody');
Query OK, 0 rows affected (0.00 sec)
mysql> set password for 'root'@'127.0.0.1' = PASSWORD('DoNotTell$AnyBody');
Query OK, 0 rows affected (0.00 sec)
Make sure you are able to login to MySQL using the new password as shown below.
# mysql -u root
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.0.77 Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
yum grouplist displays all package groups that are available in the repository. As shown below, mysql package group is called “MySQL Database”.
# yum grouplist | grep -i mysql
MySQL Database
2. What is bundled in the “MySQL Database” group?
yum groupinfo displays all the packages that are bundled in a group. This displays the mandatory, default and optional packages that are available in that particular group.
As shown below, “MySQL Database” group contains 1 mandatory package, 6 default packages, and 5 optional packages.
# yum groupinfo "MySQL Database"
Group: MySQL Database
Description: This package group contains packages useful for use with MySQL.
Mandatory Packages:
mysql
Default Packages:
MySQL-python
libdbi-dbd-mysql
mysql-connector-odbc
mysql-server
perl-DBD-MySQL
unixODBC
Optional Packages:
mod_auth_mysql
mysql-bench
mysql-devel
php-mysql
qt-MySQL
3. Install the “MySQL Database” group using yum groupinstall
yum groupinstall will install the “MySQL Database” group of packages as shown below.
# yum groupinstall "MySQL Database"
Resolving Dependencies
Dependencies Resolved
Transaction Summary
=========================
Install 12 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Installed:
MySQL-python.i386 0:1.2.1-1 libdbi-dbd-mysql.i386 0:0.8.1a-1.2.2
mysql.i386 0:5.0.77-4.el5_4.2 mysql-connector-odbc.i386 0:3.51.26r1127-1.el5
mysql-server.i386 0:5.0.77-4.el5_4.2 perl-DBD-MySQL.i386 0:3.0007-2.el5
unixODBC.i386 0:2.2.11-7.1
Dependency Installed:
libdbi.i386 0:0.8.1-2.1 libdbi-drivers.i386 0:0.8.1a-1.2.2
libtool-ltdl.i386 0:1.5.22-7.el5_4
mx.i386 0:2.0.6-2.2.2 perl-DBI.i386 0:1.52-2.el5
Complete!
Note: If you are having some issues during the installation, verify the full mysql install log to see what you are missing.
4. Verify MySQL Installation
Execute rpm -qa, to confirm that the mysql related packages are installed.
# rpm -qa | grep -i mysql
MySQL-python-1.2.1-1
mysql-5.0.77-4.el5_4.2
mysql-connector-odbc-3.51.26r1127-1.el5
mysql-server-5.0.77-4.el5_4.2
libdbi-dbd-mysql-0.8.1a-1.2.2
perl-DBD-MySQL-3.0007-2.el5
Check the /etc/passwd and /etc/group to make sure it has created a mysql username and group.
# grep mysql /etc/passwd
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
# grep mysql /etc/group
mysql:x:27:
5. MySQL Post installation – Execute mysql_install_db
mysql_install_db program will setup the necessary grant tables. The mysql_install_db program gets executed as part of the rpm installation. But, it doesn’t hurt to execute the mysql_install_db program again to make sure the grant tables are setup properly.
# /usr/bin/mysql_install_db --user=mysql
Installing MySQL system tables...OK
Filling help tables...OK
.....
The latest information about MySQL is available on the web at http://www.mysql.com
6. Start MySQL Server
# service mysqld status
mysqld is stopped
# service mysqld start
Starting MySQL: [ OK ]
7. Verify that the MySQL server is up and running.
# /usr/bin/mysqladmin version
/usr/bin/mysqladmin Ver 8.41 Distrib 5.0.77, for redhat-linux-gnu on i686
Copyright (C) 2000-2006 MySQL AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license
Server version 5.0.77
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 39 sec
Threads: 1 Questions: 2 Slow queries: 0 Opens: 12 Flush tables: 1
Open tables: 6 Queries per second avg: 0.051
# /usr/bin/mysqlshow
+--------------------+
| Databases |
+--------------------+
| information_schema |
| mysql |
| test |
+--------------------+
# /usr/bin/mysqlshow mysql
Database: mysql
+---------------------------+
| Tables |
+---------------------------+
| columns_priv |
| db |
| func |
| help_category |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
Stop and start the mysql server again to make sure they are no issues.
# service mysqld stop
Stopping MySQL: [ OK ]
# service mysqld start
Starting MySQL: [ OK ]
8. Change the MySQL root account password
Change the MySQL root account password to something secure.
# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.0.77 Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> select host, user from mysql.user;
+-----------+------+
| host | user |
+-----------+------+
| 127.0.0.1 | root |
| localhost | |
| localhost | root |
+-----------+------+
5 rows in set (0.00 sec)
mysql> set password for 'root'@'localhost' = PASSWORD('DoNotTell$AnyBody');
Query OK, 0 rows affected (0.00 sec)
mysql> set password for 'root'@'127.0.0.1' = PASSWORD('DoNotTell$AnyBody');
Query OK, 0 rows affected (0.00 sec)
Make sure you are able to login to MySQL using the new password as shown below.
# mysql -u root
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.0.77 Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
Wednesday, April 28, 2010
File System use for Linux
Ext3
Ext3 is a journaling filesystem developed by Stephen Tweedie. It is compatible to ext2 filesystems; actually you can look at it as an ext2 filesystem with a journal file. The journaling capability means no more waiting for fsck’s or worrying about metadata corruption. What is most noticeable is that you can switch back and forth between ext2 and ext3 on a partition without any problem: it is just a matter of giving the mount command the right filesystem type.
Ext3 FAQ
JFS
The Journaled File System (JFS) provides a log-based, byte-level file system that was developed for transaction-oriented, high performance systems. Scalable and robust, its advantage over non-journaled file systems is its quick restart capability: JFS can restore a file system to a consistent state in a matter of seconds or minutes.
While tailored primarily for the high throughput and reliability requirements of servers ( from single processor systems to advanced multi-processor and clustered systems), JFS is also applicable to client configurations where performance and reliability are desired.
JFS is IBM software.
JFS Overview at IBM Developer Works
ZFS
ZFS is a new kind of file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability.
ZFS includes a pooled storage model, is always consistent on disk, offers protection from data corruption, does live data scrubbing, makes instantaneous snapshots and clones, has fast native backup and restore capabilities, is highly scalable, has built in compression and a simplified administration model.
ZFS is shipped with OpenSolaris.
OpenSolaris Community: ZFS
XFS
XFS is a high-performance journaling filesystem. XFS combines advanced journaling technology with full 64-bit addressing and scalable structures and algorithms. This combination delivers the most scalable high-performance filesystem ever conceived.
XFS is SGI software.
XFS homepage at SGI Developer Central
VXFS
VXFS is the Veritas Filesystem and is distributed through Symantec, who bought Veritas quite a while ago.
Symantec Homepage
Remote
NFS
The Network Filesystem is available on every UNIX-like OS.
Wikipedia about NFS
Open AFS
AFS is a distributed filesystem product, pioneered at Carnegie Mellon University and supported and developed as a product by Transarc Corporation (now IBM Pittsburgh Labs). It offers a client-server architecture for file sharing, providing location independence, scalability, security, and transparent migration capabilities for data.
IBM branched the source of the AFS product, and made a copy of the source available for community development and maintenance. They called the release OpenAFS.
OpenAFS Homepage
CIFS/SAMBA
Samba is an Open Source implementation of Microsoft’s Common Internet Filesystem (CIFS).
CIFS itself is part of all Microsoft Windows versions.
Samba support for IRIX can be bought from SGI itself.
Samba Homepage
CIFS or Public SMB Information on Common Internet File System
Samba for IRIX FAQ
Distributed
Lustre
Lustre is a scalable, secure, robust, highly-available cluster file system. It is designed, developed and maintained by Cluster File Systems, Inc.
The central goal is the development of a next-generation cluster file system which can serve clusters with 10,000′s of nodes, petabytes of storage, move 100′s of GB/sec with state of the art security and management infrastructure.
Lustre runs today on many of the largest Linux clusters in the world, and is included by CFS’s partners as a core component of their cluster offering (examples include HP StorageWorks SFS, and the Cray XT3 and XD1 supercomputers). Today’s users have also demonstrated that Lustre scales down as well as it scales up, and run in production on clusters as small as 4 and as large as 15,000 nodes.
The latest version of Lustre is always available from Cluster File Systems, Inc. Public Open Source releases of Lustre are made under the GNU General Public License. These releases are found here, and are used in production supercomputing environments worldwide.
You may subscribe to the lustre-announce mailing list to be informed of releases.
Lustre development would not have been possible without funding and guidance from many organizations, including several US National Laboratories, early adopters, and product partners.
Lustre Homepage
GPFS
The IBM General Parallel File System (GPFS) is a high-performance shared-disk file system that can provide fast, reliable data access from all nodes in a homogenous or heterogenous cluster of IBM UNIX® servers running either the AIX 5L or the Linux operating system.
GPFS allows parallel applications simultaneous access to a set of files (or even a single file) from any node that has the GPFS file system mounted while providing a high level of control over all file system operations.
GPFS is IBM software.
GPFS Homepage at IBM
GFS
GFS (Global File System) is a cluster file system. It allows a cluster of computers to simultaneously use a block device that is shared between them (with FC, iSCSI, NBD, etc…). GFS reads and writes to the block device like a local filesystem, but also uses a lock module to allow the computers coordinate their I/O so filesystem consistency is maintained. One of the nifty features of GFS is perfect consistency — changes made to the filesystem on one machine show up immediately on all other machines in the cluster.
GFS Homepage at Red Hat
PVFS
The goal of the Parallel Virtual File System (PVFS) Project is to explore the design, implementation, and uses of parallel I/O. PVFS serves as both a platform for parallel I/O research as well as a production file system for the cluster computing community. PVFS is currently targeted at clusters of workstations, or Beowulfs.
PVFS supports the UNIX I/O interface and allows existing UNIX I/O programs to use PVFS files without recompiling. The familiar UNIX file tools (ls, cp, rm, etc.) will all operate on PVFS files and directories as well. This is accomplished via a Linux kernel module which is provided as a separate package.
PVFS stripes file data across multiple disks in different nodes in a cluster. By spreading out file data in this manner, larger files can be created, potential bandwidth is increased, and network bottlenecks are minimized. A 64-bit interface is implemented as well, allowing large (more than 2GB) files to be created and accessed.
The Parallel Virtual Filesystem Project
PVFS 2
Ext3 is a journaling filesystem developed by Stephen Tweedie. It is compatible to ext2 filesystems; actually you can look at it as an ext2 filesystem with a journal file. The journaling capability means no more waiting for fsck’s or worrying about metadata corruption. What is most noticeable is that you can switch back and forth between ext2 and ext3 on a partition without any problem: it is just a matter of giving the mount command the right filesystem type.
Ext3 FAQ
JFS
The Journaled File System (JFS) provides a log-based, byte-level file system that was developed for transaction-oriented, high performance systems. Scalable and robust, its advantage over non-journaled file systems is its quick restart capability: JFS can restore a file system to a consistent state in a matter of seconds or minutes.
While tailored primarily for the high throughput and reliability requirements of servers ( from single processor systems to advanced multi-processor and clustered systems), JFS is also applicable to client configurations where performance and reliability are desired.
JFS is IBM software.
JFS Overview at IBM Developer Works
ZFS
ZFS is a new kind of file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability.
ZFS includes a pooled storage model, is always consistent on disk, offers protection from data corruption, does live data scrubbing, makes instantaneous snapshots and clones, has fast native backup and restore capabilities, is highly scalable, has built in compression and a simplified administration model.
ZFS is shipped with OpenSolaris.
OpenSolaris Community: ZFS
XFS
XFS is a high-performance journaling filesystem. XFS combines advanced journaling technology with full 64-bit addressing and scalable structures and algorithms. This combination delivers the most scalable high-performance filesystem ever conceived.
XFS is SGI software.
XFS homepage at SGI Developer Central
VXFS
VXFS is the Veritas Filesystem and is distributed through Symantec, who bought Veritas quite a while ago.
Symantec Homepage
Remote
NFS
The Network Filesystem is available on every UNIX-like OS.
Wikipedia about NFS
Open AFS
AFS is a distributed filesystem product, pioneered at Carnegie Mellon University and supported and developed as a product by Transarc Corporation (now IBM Pittsburgh Labs). It offers a client-server architecture for file sharing, providing location independence, scalability, security, and transparent migration capabilities for data.
IBM branched the source of the AFS product, and made a copy of the source available for community development and maintenance. They called the release OpenAFS.
OpenAFS Homepage
CIFS/SAMBA
Samba is an Open Source implementation of Microsoft’s Common Internet Filesystem (CIFS).
CIFS itself is part of all Microsoft Windows versions.
Samba support for IRIX can be bought from SGI itself.
Samba Homepage
CIFS or Public SMB Information on Common Internet File System
Samba for IRIX FAQ
Distributed
Lustre
Lustre is a scalable, secure, robust, highly-available cluster file system. It is designed, developed and maintained by Cluster File Systems, Inc.
The central goal is the development of a next-generation cluster file system which can serve clusters with 10,000′s of nodes, petabytes of storage, move 100′s of GB/sec with state of the art security and management infrastructure.
Lustre runs today on many of the largest Linux clusters in the world, and is included by CFS’s partners as a core component of their cluster offering (examples include HP StorageWorks SFS, and the Cray XT3 and XD1 supercomputers). Today’s users have also demonstrated that Lustre scales down as well as it scales up, and run in production on clusters as small as 4 and as large as 15,000 nodes.
The latest version of Lustre is always available from Cluster File Systems, Inc. Public Open Source releases of Lustre are made under the GNU General Public License. These releases are found here, and are used in production supercomputing environments worldwide.
You may subscribe to the lustre-announce mailing list to be informed of releases.
Lustre development would not have been possible without funding and guidance from many organizations, including several US National Laboratories, early adopters, and product partners.
Lustre Homepage
GPFS
The IBM General Parallel File System (GPFS) is a high-performance shared-disk file system that can provide fast, reliable data access from all nodes in a homogenous or heterogenous cluster of IBM UNIX® servers running either the AIX 5L or the Linux operating system.
GPFS allows parallel applications simultaneous access to a set of files (or even a single file) from any node that has the GPFS file system mounted while providing a high level of control over all file system operations.
GPFS is IBM software.
GPFS Homepage at IBM
GFS
GFS (Global File System) is a cluster file system. It allows a cluster of computers to simultaneously use a block device that is shared between them (with FC, iSCSI, NBD, etc…). GFS reads and writes to the block device like a local filesystem, but also uses a lock module to allow the computers coordinate their I/O so filesystem consistency is maintained. One of the nifty features of GFS is perfect consistency — changes made to the filesystem on one machine show up immediately on all other machines in the cluster.
GFS Homepage at Red Hat
PVFS
The goal of the Parallel Virtual File System (PVFS) Project is to explore the design, implementation, and uses of parallel I/O. PVFS serves as both a platform for parallel I/O research as well as a production file system for the cluster computing community. PVFS is currently targeted at clusters of workstations, or Beowulfs.
PVFS supports the UNIX I/O interface and allows existing UNIX I/O programs to use PVFS files without recompiling. The familiar UNIX file tools (ls, cp, rm, etc.) will all operate on PVFS files and directories as well. This is accomplished via a Linux kernel module which is provided as a separate package.
PVFS stripes file data across multiple disks in different nodes in a cluster. By spreading out file data in this manner, larger files can be created, potential bandwidth is increased, and network bottlenecks are minimized. A 64-bit interface is implemented as well, allowing large (more than 2GB) files to be created and accessed.
The Parallel Virtual Filesystem Project
PVFS 2
Subscribe to:
Posts (Atom)