Thursday, July 31, 2008

Securtiy with IPTABLES

Another way to secure your server is by using iptables, you can use iptables together with tcp_wrappers or on their own, the choice is yours, the advantage with iptables is that iptables can be used to ACCEPT, DROP or REJECT packets of Data, it can also be used to FORWARD specific Data onto different Destinations and it can be used to configure NAT (Network Address Translation) also known as Masquerading.

the usage of IPTABLES is as follows:

iptables -t type (action) (direction) (type of packet) -j (what to do)

their are two types that you can choose (the -t switch)
filter = sets a rule for filtering packets
nat= configures Network Address Translation, also known as Masquerading
the default type is filter , if you don't specify a -t type the iptables command will assume that you are trying to setup a filtering rule. so you can leave out the -t switch if you are setting up a firewall rule.
next is the (action)
you can either
-A append a rule
-D delete a rule
-L list the currently configured Rules
-F flush the rules

next you need to specify which packets are the rules applied to (direction of packet)
INPUT = all incoming packets
OUTPUT = all outgoing packets
FORWARD = all packets that are being forwarded to another computer.

next you need to specify the source or destination address of the packet
-s ipaddress
-d ipaddress

next you need to specify the protocol of the packet using the -p switch
eg
-p tcp and then the port using the --dport switch eg
-p tcp --dport 80
and then finally what needs to be done with the packet which is the -j switch
DROP = the packet is dropped (no message is sent to the requesting host)
REJECT = the packet is rejected and an error message is sent to the requesting host
ACCEPT = the packet is Accepted
an ACCEPTED Packet can be forwarded by using the - A switch and then FORWARD
lets set up an iptable chain.
The first step is always to see what iptables are already configured . type
iptables -L
this reads the iptables from your /etc/sysconfig/iptables file (we do not edit this file directly it is best to use the iptables command with the relevant switches to configure your chains.)
iptables -L will return your rules in three different categories INPUT,FORWARD and OUTPUT
the following command will set a rule that denies all traffic from 192.168.0.0 network
iptables -A INPUT -s 192.168.0.0/24 -j REJECT
the following rule will make your server un-ping-able as it will drop all ICMP (ping) packets. Assume that your network is 192.168.0.0 the (!) inverts the meaning in this case the command applies to all IP addresses except those on the 192.168.0.0 network

if you need to insert the rule at line number 3 of the chain then type
iptables -I INPUT 3 -s 192.168.0.0/24 -j REJECT


iptables -A INPUT -s !192.168.0.0/24 -p icmp -j DROP
to delete any of the above commands simply retype them and change the -A to a -D eg:
iptables -D INPUT -s !192.168.0.0/24 -p icmp -j DROP will remove the previous chain
you can check your progress by typing iptables -L at anytime.
Once you have added the iptable rules that you want , you need to save your configuration. This is done with the following command
/etc/init.d/iptables save
this will save your configuration into the /etc/sysconfig/iptables file
you also need to insure that iptables starts up on run levels 2,3,4 and 5 so that it is persistent after a reboot
to do this type the following
chkconfig iptables on

Security with TCP wrappers

TCP_wrappers is on by default and you do not need to start any service for it to work.
TCP_wrappers is configured by editing 2 files /etc/hosts.allow and /etc/hosts.deny

When your system receives a network request for a service. The request is passed on to tcp_wrappers
tcp_wrappers is very straight forward and easy to set-up.
Users and clients that are listed in /etc/hosts.allow are allowed access to the listed services
and users and clients that are listed in the /etc/hosts.deny file are denied access to the listed services.
It's important to know the order of things that your system takes to make its decisions. When a request is made of your system, your system will first read your /etc/hosts.allow file and if it finds a rule in there for the requested service the rule is obeyed and no additional searches take place. If there are no rules in /etc/hosts.allow for the requested service then your system will look in /etc/hosts.deny and if it sees a rule in their for the service the service is denied. If your system sees no rules in neither /etc/host.allow nor in /etc/hosts.deny then the service is automatically granted access.
the syntax of your access rules are as follows

(SERVICES to allow or block separated by commas) : Clients or source destinations
so lets set up some rules, edit /etc/host.deny using your favourite text editor and add the following line

ALL : ALL

this will make your server air tight as every service from every host is Denied.
however we can allow the clients and services that we want by adding them to the /etc/hosts.allow file, remember your system Will first check your /etc/hosts.allow file and if it finds any rules in there then those rules are obeyed and no further checking for those rules will take place , so by adding the following line to /etc/hosts.allow

ALL : 192.168.0.0/24

substitute 192.168.0.0/24 with the ip address of the network that you want to allow access to your server from


this will allow access to all services from the 192.168.0.0 network to have access on your server but all other networks will be denied since the rule in your /etc/hosts.deny file will block them.

Your access rules can be very flexible , for example you could add a rule like so into your /etc/hosts.allow file

ALL : 192.168.0.0/24 EXCEPT 192.168.0.10

this would allow all hosts from the 192.168.0.0 network access to all services on your server except for host 192.168.0.10 who will be denied.

you can also allow access to specific services only eg:

sshd, ftpd, telnetd, http : 192.168.0.20

would allow host 192.168.0.20 to ssh, ftp telnet and access your server over http .

likewise you could also deny access to specific services and specific users by adding the rules into your /etc/hosts.deny file

eg :
ALL EXCEPT sshd : 192.168.0.0/24
added to your /etc/hosts.deny file would deny all services except ssh from all hosts on the 192.168.0/24 network.

as you can see TCP_WRAPPERS is extremely flexible and straight forward to use.

Other recognised commands that you can put into your /etc/hosts.allow and /etc/hosts.deny files are
.hostname.com (will block or allow clients from the specified hostname eg :
ALL : hostname.com in your /etc/hosts.allow file will allow all clients from the hostname.com domain access to all services on your server.

user@machine_name.hostname.com will apply to the specific user from a given computer

192.168. since the IP address ends with a . it specifies all hosts whose IP address starts with 192.168.

to see the exact names of all the services that you can allow or deny take a look at your /etc/services file.

Running Commands Conditionally

in Linux every command you run produces and records an exit status on whether the command was successful or not. If a command is successful the exit status will be recorded as "0" and if the command is not successful then the exit status is recorded as any number between "1 and 255". The exit status of the last command that you ran is recorded inside the $? variable.
To see what the exit status was of your previous command simply type echo $?
were this is useful is it allows you to run commands conditionally, based on the commands exit status. When you type two commands and you separate them with a && the second command will only run if the first command produced an exit status of "0" ie was successful.
If you separate the commands with a || then the second command will only run if the first command produced an exit status of "1-255" ie was not successful.
eg
ping 192.168.0.1 -c1 -w2 && echo "host is up"

- c1 = send 1 ping packet
-w2 = wait 2 seconds for a response

will display "host is up" if the ping command was successful at reaching the host at 192.168.0.1 and
ping 192.168.0.1 -c1 -w2 || echo "host is down"

will display "host is down" if the ping command was unsuccessful in reaching the host at 192.168.0.1

to get the desired result you should combine the commands like so

ping 192.168.0.1 -c1 -w2 && echo "host is up" || echo "host is down"

since you don't care about the actual output of the ping command (you are only interested if it was succesful or not ) You can redirect stdout and stderror to /dev/null so your final command would be something like:
ping 192.168.0.1 -c1 -w2 &> /dev/null && echo "Host is up" || echo "Host is Down"

Sunday, July 27, 2008

Save, Convert and join Youtube movies for playback on your Blackberry

lets say you see 4 movie clips on you tube that you want to join together to form 1 movie and save it to and play it on your Blackberry.
to start first download the mencoder tool. It should be available in your distributions repository.
type apt-get install mencoder if you have a debian derivative distro or
type yum install mencoder if you have a red hat one.


When you watch a movie clip on you tube the .flv file is automatically saved into your /tmp folder so once you have watched a clip that you want, you can simply copy it from your /tmp folder and save it somewhere safe, youtube saves its files with a name that starts with Flash followed by some arbitrary characters like FlashFgna. Copy these files to another folder so that they do not get deleted as your system will delete all files in your /tmp folder when you log out. so for my example watch the four movies you want on you tube, each one in a different tab of your browser once you have watched all 4 movies, open up a terminal window
type cd ~ then type mkdir videos which will create the directory were we are going to be working in.

cd videos to cd into the folder
type copy /tmp/Flash* .
will copy the four .flv files to your /home/username/videos/ folder don't worry if you don't see the .flv extension they are .flv files Linux doesn't care for extensions.

Sometimes you can skip the following step and you can try and join and convert your Flash files from .flv to .mp4. using the mencoder tool. but I have been much more successfull first encoding the files to the .avi format and then if i want to put the file on my Blackberry i will encode from .avi to .mp4



to encode from Flash to .avi we need to create the tool that will convert flv to avi files, here is a script that can do this.

Copy the following lines into your clipboard by highlighting them directly from this post and hitting cntl- "c"

#!/bin/sh

if [ -z "$1" ]; then
echo "Usage: $0 {-divx|-xvid} list_of_flv_files"
exit 1
fi

# video encoding bit rate
V_BITRATE=1000

while [ "$1" ]; do
case "$1" in
-divx)
MENC_OPTS="-ovc lavc -lavcopts \
vcodec=mpeg4:vbitrate=$V_BITRATE:mbd=2:v4mv:autoaspect"
;;
-xvid)
MENC_OPTS="-ovc xvid -xvidencopts bitrate=$V_BITRATE:autoaspect"
;;
*)
if file "$1" | grep -q "Macromedia Flash Video"; then
mencoder "$1" $MENC_OPTS -vf pp=lb -oac mp3lame \
-lameopts fast:preset=standard -o \
"`basename $1 .flv`.avi"
else
echo "$1 is not Flash Video. Skipping"
fi
;;
esac
shift
done


type vi /usr/local/bin/flv2avi.sh
(/usr/local/bin is a good place to save scripts as it is part of your path environment which means you will be able to execute your script from anywhere on your system)

this will open up your vi editor
once open type "i" to go into insert mode
then click on edit paste to paste the code into your script.

type :wq to save your script and exit out of vi

type chmod 755 /usr/local/bin/flv2avi.sh to make your script executable

to convert all the .flv files to avi we can do them all in one command. Type flv2avi.sh -divx Flashfile1 Flashfile2 Flashfile3 Flashfile4

were file1 file2 file3 file4 are the .flv files you want to convert.
once done you will have 4 additional files in your /home/username/videos folder all with .avi extensions.

now to join these files together and convert them to one .mp4 file which is the format that works best on your blackberry we also do it all in one command
mencoder file1.avi file2.avi file3.avi file4.avi -o newfilename.mp4 -ovc lavc -oac lavc
this will join all 4 files into one file and convert it to a .mp4 file
next just copy the file onto your blackberry and you will be able to play it using your blackberry's media player.

Saturday, July 26, 2008

Multiple Terminals in one Terminal Window with Terminator.

After a very short while of working on my desktop I often find myself with a mass of open terminal windows, and it sometimes becomes difficult to work as I struggle to find the Terminal Window that I want. I don't particularly like to use tabbed terminals to open multiple terminals, as I often need to see all of my open terminals side by side at the same time. Terminator to the Rescue!
Terminator is a virtual terminal program that allows you to split multiple terminal Windows in the same window. Terminator should be available in your Linux distributions Repository. apt-get install terminator if you use a Debian Flavoured Distribution or yum install terminator if you use a Red Hat one.
Once Installed you will find the terminator icon to start the program under Applications, accessories. or you can start terminator at the command line by typing terminator &
When it starts, a Terminal Window will open which will allow you to open new Terminals within your terminal in a split screened environment. What I like about this is when you minimize your terminal and you want to get back to it just by maximizing one Terminal you have all the terminals you were working on immediately accessible.
Once open try the following
Ctrl+Shift+O
Split terminals Horizontally.
Ctrl+Shift+E
Split terminals Vertically.
Ctrl+Shift+N
Move to next terminal.
Ctrl+Shift+P
Move to previous terminal.
Ctrl+Shift+W
Close the current terminal.
Ctrl+Shift+Q
F11 toggle Full Screen
you can also use your mouse to switch between terminals and to resize them by dragging their borders to the required size.

Friday, July 25, 2008

Session Managment with Screen

You log into your remote server via SSH and are busy downloading and installing a new program. In the middle of the download you loose your connection to your server. "Connection Closed" You have just lost your session! Screen to the rescue.

Screen is a window manager for your ssh terminal sessions. Screen is an absolute life saver when working over ssh as it allows you to reconnect to your sessions and continue working exactly were you left off. Screen allows you to re-attach to your session.

Screen is available in your Linux distributions repository. To install screen type

apt-get install screen if you are using a Debian based distro. Or yum install screen if you are using a red hat derivative distribution. You will want to run and install screen onto the machine that you are connecting to. Once installed start screen by typing screen.

If you are presented with a text message just hit enter. If nothing happens don't worry it just means that you are now inside a window within screen and it is running and working.

Screen uses the command "Ctrl-A" to send commands to screen instead of the shell. To get help, just type "Ctrl-A" then "?"

Screen supports multiple windows. This is useful for doing simultaneous tasks on the same machine over ssh without opening new sessions. Sometimes I need to run multiple tasks on the same remote machine or whilst one task is busy running I need to start up another task. Without screen I would need to make a new connection to the same machine or even more multiple connections and if any of the connections drop then i am screwed . With screen you connect to your remote machine only once and can run multiple tasks on the same connection and if your connection breaks, no problem you simply reconnect and re-attach your session.

To open a new window, you just use "Ctrl-A" "c"

run your task, eg mtr www.google.com

Now open a new window with "Ctrl-A" "c" again and start another task this time lets start top type top

To get back to your previous screen (mtr www.google.com task), use "Ctrl-A "n"

You can create multiple windows and toggle through them with "Ctrl-A" "n" for the next screen or "Ctrl-A" "p" for the previous one .

If you want to close your session but want to return to it later then you must detach from your session instead of closing it. This will leave your process running and will allow you to re-attach to the same process later. "Ctrl-A" "d". This will drop you into your shell. All screen windows are still there and you can re-attach to them later.

So you are using screen now and busy downloading a new program and suddenly your connection drops. Don't worry screen will keep the download going. Login to your system and type

screen - R to re-attach to your session and then use Ctrl - A "n" and "p" to toggle between all the sessions you were running on the remote system before your connection was lost.

another useful feature of screen is its ability to monitor a window for activity or for silence.

lets say you are downloading a file and you want to know when the download is finished, you will need to monitor for silence on that screen. To do that type "Ctrl-A" "_" . When your download is complete you will get an alert at the bottom with the window number. To quickly go to that window, use "Ctrl-A" " . After you do this, just type in the number of the window and enter. To stop monitoring, go to that window and undo the monitor with the same command. To monitor for activity type Ctrl-A "M" this will alert you when something new appears on the session that you wanted to monitor.

Screen can also be used to share a terminal session with another user. This is very useful if you need to show someone how to do something.

The host starts screen in a local xterm, using the command screen -S SessionName. The -S switch gives the session a name, which makes multiple screen sessions easier to manage.
Type:
screen -S screendemo
The remote user (bwayne) uses SSH to connect to the host computer (cgerada).
Type:
ssh bwayne@cgerada.computer.ip.address
The host (cgerada) then has to allow multiuser access in the screen session via the command CTRL-A :multiuser on .
Type:
CTRL-A
:multiuser on
Next, the host (cgerada) must grant permission to the remote user (bwayne) to access the screen session using the command CTRL-A :acladd user_name where user_name is the remote user's login ID.
Type:
CTRL-A
:acladd bwayne
The remote user can now connect to the hosts 'screen' session. The syntax to connect to another user's screen session is screen -x host_username/sessionname.
Type:
screen -x cgerada/screendemo

Voila both users will now share the same terminal session.

How to convert Multiple Images for emailing

Imagemagick is normally included with most linux distributions, but if it isn't then you can install it by typing the following
apt-get install imagemagick for debian based distributions
or yum install imagemagick for red hat derivative distributions.

imagemagick is an awesome and extremely powerful image manipulation tool. For more detailed and other uses of this great tool read the man pages ie. man convert or visit their website http://www.imagemagick.org.

I use imagemagic to quickly resize a large batch of jpg files so that they are a suitable size for emailing

cd into the directory that you have your images saved, the ones you want to resize

then type the following

for k in $(ls *.jpg); do convert $k -resize 800x600 re_$k; done

replace 800x600 with whatever size you want to resize your images to. eg 640x480 or 320x200 etc this command will go through your current directory and will convert all your .jpg files by creating new files, saving the new converted files to a new filename with re_ infront of the existing file name. The original files are left intact.

you can also convert your pictures to different formats ie
convert image.jpg image.bmp
or convert image.gif image.tiff
or convert image.png image.jpg
will convert between the different formats

you can also use it to convert to a pdf file or to convert a pdf file to a jpg
eg convert file.jpg file.pdf or convert file.pdf file.jpg

Wednesday, July 23, 2008

How to Mount an ISO or an MDF image

The mount command can be used to mount any thing from a samba / nfs share to an ISO image.

in windows you need such tools as Daemon tools , Alcohol 120 etc to accomplish the same. In Linux it's a lot simpler. ISO images can be mounted without any additional tools. MDF images can also be mounted but they first need to be converted to an ISO file. To convert an MDF image to an ISO image use the mdf2iso tool which you can download from your distributions repository. eg yum install mdf2iso if you use a Red Hat derivative distro. Or apt-get install mdf2iso if you use a debian one. Once you convert the MDF file to an ISO you simply use the mount command to mount the image.

mount -t iso9660 -o loop

the loop option mounts the iso as a loopback device. This is neccassery when you are mounting a file of another file system in this case an iso file

Example:

mkdir /mnt/image

mount -t iso9660 -o loop /home/cgerada/films/the_dark_Knight.iso /mnt/image

The ‘-t’ option specifies the file type - this is optional, Linux is intelligent enough to see that the image you are mounting is of the iso9660 file type so you could just as well type.

mount -o loop /home/cgerada/films/the_dark_knight.iso /mnt/image

then cd into the /mnt/image directory to use the files ie cd /mnt/image.

To convert an MDF (alcohol 120) file to an ISO

type mdf2iso filename.mdf newfilename.iso

and then mount the ISO file.

Saturday, July 19, 2008

How to Upgrade your full distribution of Ubuntu to the latest version

For your server that doesn't have a gui
sudo -i
enter your root password
apt-get install update-manager-core
do-release-upgrade
this will do a complete update to the latest version of Ubuntu, including updating of your sources list

on a desktop installation were you have Gnome or KDE installed press ALT -F2 then type
update-manager --devel-release and then click on run
this will also do a complete upgrade.

Friday, July 18, 2008

Look from 15 ft away

When you look at this picture up close you see its Albert Einstein.

But if you stand 15 feet away,

It will become Marilyn Monroe.



Give it a try

Wednesday, July 9, 2008

How to deny users access to certain sites at specific times

right, now you've got Squid up and running and its saving your company oodles of bandwidth as it serves cached websites to your users. other great features of squid is its use of Access lists (ACL's). Here is a way that you can restrict your users access to specific sites during specific times, lets say for this example that you do not want your users to access youtube, facebook and myspace between 8:30 am and 5:00pm but you didn't mind if they accessed those sites before 8:30am and after 5:00pm one way of accomplishing this is the following.
create a file in your /etc/squid/ directory called band_sites
you can do this by typing vi /etc/squid/band_sites
then in the empty file put a list of sites you want to ban your users from visiting
like so
myspace.com
facebook.com
youtube.com

save the file by typing :wq
change the ownership of the file so that the squid group can access the file
chown root.squid /etc/squid/band_sites
and then change the permissions to the file
chmod 640 /etc//squid/band_sites

edit your squid configuration file and then add the following ACL's.
It is important that you put these ACL's at the top
of the file, before your http_access allow our_networks
since ACL's are applied from top to bottom.
vi /etc/squid/squid.conf
acl work_time time 8:30-17:30
acl band_sites dstdom_regex -i "/etc/squid/band_sites"
http_access deny band_sites work_time

save your config file
and then reload squid by typing
/etc/init.d/squid reload

Done

Squid Proxy quick and easy

the Squid config file can be quite daunting as it has over 4000 lines.

here's how to get squid up and running quickly.

Before activating squid you need to add three lines to the config file.
You need to add the hostname of your squid server. To do this add the following line to your
/etc/squid/squid.conf file , add it near the associated comment in the file.
were hostname.domainname.com is the host name of your squid proxy server.

visible_hostname hostname.domainname.com

Next you need to add the following basic access lists, so that your machines will be able to connect to squid.
acl our_networks src 192.168.0.0/24
http_access allow our_networks

were 192.168.0.0/24 is the address's of the machines that will be accessing your proxy server. You can add as many address ranges as you want. eg
acl our_networks 192.168.0.0/24 172.16.0.0/24 10.0.0.0/24
etc etc. save your changes and exit.
Make sure that you allow port 3128 on your firewall, as that is the default port that squid uses. You can change this port to anything you want like 8080 by editing the default port stanza in your /etc/squid/squid.conf file

type squid -z
this will create the squid swap directories.

Make sure that squid starts up on start up by typing
chkconfig squid on this will insure it starts on run levels 3 and 5

and then start squid by typing /etc/init.d/squid start

Done, squid should be up and running and caching webpages for your users.

Squid is a high performance very flexible and powerful caching proxy server
with many configurable options , for example, you can set up
password protected web browsing authentication for your users,
so that you can monitor web usage associated to login name, not only IP address. This is useful in an environment were more than one user uses the same machine. You can also restrict access to websites at certain times of the day for certain users.A comprehensive user guide on squid is available at http://euler.aero.iitb.ac.in/docs/web/squid/html/book1.html

Tuesday, July 8, 2008

Apache , how to password protect your site

You can limit access to your websites to authorised users with passwords, these credentials can be different and independent of the regular users of your web server.

edit your /etc/httpd/conf/httpd.conf file and add the following stanza
were the first line is the path of the directory you want to protect

</Directory"/var/www/html/website/">
AuthName "any Text in here you want to be displayed"
AuthUserFile /etc/httpd/webpass
Require valid-user
< /Directory >

once you have edited the file you will need to restart or reload the httpd daemon do this by typing
/etc/init.d/httpd reload
next type the following
htpasswd -c /etc/httpd/webpass username
were username is the username you want to allow access to the site
you will be prompted for a password for the username you entered and the information will be saved into the /etc/httpd/webpass file in an encrypted format, so even if someone reads that file they will not be able to make out what the password is.
If you want to add more authorised users to the file simply use the same command but without the -c option
-c was to create the original file , since it has now been created you just want to add users so you don't use the -c
eg htpasswd /etc/httpd/webpass username2when you access the site you will be prompted for a user name and password.

Monday, July 7, 2008

Logical Volumes LVM

Logical Volumes are the greatest thing since sliced bread
if your data is stored on a logical volume and you run out of disk space, you can simply add more hard drives to your system and then grow your logical volume onto the new physical hard drives giving your data the extra space it needs, logical volumes can also work on top of Raid since logical volumes by them selves gives you no redundancy at all it is a good idea to have them setup on top of a Raid array.

To set up a logical volume partitions must first be converted into physical volumes using the pvcreate command. once you have added your new hard drives to the system, use fdisk to tag the drives as LVM's the tag hex code for LVM is 8e once tagged you can convert the drives to physical volumes to convert /dev/hda and /dev/hdb into physical volumes type:

pvcreate /dev/hda
pvcreate /dev/hdb

NOTE : to setup a LVM on top of a linux software Raid array you simply convert your software raid device to a physical volume eg:
pvcreate /dev/md0

Once you have one or more physical volumes created, you can create a volume group from these PVs using the vgcreate command.

vgcreate vg1 /dev/hda /dev/hdb

this creates a VG called vg1 from the two disks, /dev/hda and /dev/hdb
more PVs can be added to this volume group using the vgextend command.

pvcreate /dev/hdc
vgextend vg1 /dev/hdc

this will add /dev/hdc to the volume group "vg1"
to remove the PV from vg1 use the vgreduce command

vgreduce vg1 /dev/hdc
you can add as many disks to the volume as you like, think as the volume
as your cache of storage.
Use lvcreate command to create a logical volume which will use the free capacity in the volume group pool. (ie your storage cache)

lvcreate -n lv1 -L 100G vg1

or you can specify the capacity in PE's "physical extents" to find out how many PE's are available
in your volume group type.

vgdisplay vg1 | grep "Total PE"

which returns

Total PE 34456

Then use lvcreate to create a logical volume with 34456 extents

lvcreate -n lg1 -L 34456 vg1

you can also allocate a logical volume from a specific physical partition in the volume group by specifying the partition at the end of the lvcreate command.

lvcreate -L100G -n lg1 /dev/hda /dev/hdb

A logical volume can be removed from a Volume group using lvremove
you must first umount the logical volume.

umount /dev/vg1/lg1
lvremove /dev/vg1/lg1

when you add more hard drives to your system and add them to the Volume group you can then extend the logical volume to make use of the added storage by using the lvextend command. You can specify either an absolute size for the extended LV or how much additional storage you want to add to the LV.

lvextend -L120G /dev/vg1/lg1

will extend the logical volume to 120 GB, while

lvextend -L+20G /dev/vg1/lg1

will extend the logical volume by an additional 20 GB. Once a logical volume has been extended, the file system must be expanded to use the added storage.
to do this use the resize2fs command.

resize2fs /dev/vg1/lg1your done...

other usefull commands
pvs will list all your partitions that you have assigned to a volume group
lvs will list all of your configured logical Volumes
vgs will show you how much available or free space you have available in your volume group
lvscan will scan for active logical volumes
vgdisplay will display the information of your volume group
lvdisplay will display currently configured logical volumes


For a good guide to grow a Virtual Machine checkout the following link
http://www.jaredlog.com/?p=1133

Saturday, July 5, 2008

Software raid on Linux

to create a software raid in Linux we use the mdadm command
first you will need the partitions available to you for the raid, using fdisk you must tag the partitions with the "linux raid autodetect " tag using the hex code id of fd to do this just type fdisk /dev/hda (you use the name of the partion you want to tag then type t and then type fd type w to save and then type partprobe to probe the changes into your kernel

next you need to create your raid partition , lets create a raid 1 partition which is a mirrored set and it uses 2 disks, but we are going to add a 3rd disk and use it as a spare, so if one of the 2 disks in the mirror break the spare will automatically replace it in the raid configuration and rebuild itself.

so we have 3 partitions taged with the "linux raid autodetect" tag id fd

to create a raid 1 partition we type mdadm -C /dev/md0 -l 1 -n 2 -x 3 -a yes /dev/hd{3..5}
-C = create
-l 1 = raid level 1
-n 2 = how many disks we will be using
-x 1= x stands for spare (how many spare disks we will be using
-a yes = instructs udev to create the md device file if it does not already exist
/dev/hd{3..5} are the disks you want to use in the raid you could type them separately ie /dev/hd3 /dev/hd4 /dev/hd5 if you want but /dev/hd{3..5} is shorter to type and is the exact same thing.

once done you need to format your raid partition.
mkfs.ext3 /dev/md0
next create a directory were you want to mount your raid partition
mkdir /media/raid1
lets give the partition a label we will use the label name in the fstab file this is not necessary however it is good practice to label your partitions so that if they ever change their /dev position the label will remain the same and they will still mount.
e2label /dev/md0 raid1

then we can mount it in fstab using its label name ... in your /etc/fstab file add the following line
LABEL=raid1 /media/raid5 ext3 defaults 1 2
save your fstab file and type mount-a to have your fstab file read and loaded by the system
to monitor our raid partition we can type cat /proc/mdstat
or watch -n1 cat /proc/mdstat this will update the monitoring process every second

you can even simulate a drive failure to test if your raid works,
type watch -n1 cat /proc/mdstat in a terminal window and set that window to always be on top , then open another window to fail your drive in and you can watch your raid rebuild itself in the monitoring window.
in the other window type mdadm /dev/md0 -f /dev/hd3 (this will fail drive /dev/hd3 in your raid)
(watch what happens in your monitoring window) you can then remove the failed drive from the raid set by typing mdadm /dev/md0 -r /dev/hd3
and to put a new drive back into you raid set type mdadm /dev/md0 -a /dev/hd3