Sunday, November 23, 2008

Compiling Linux Kernel 2.6.27

How to: Compile Linux kernel 2.6.27

Compiling custom kernel has its own advantages and disadvantages. However, new Linux user / admin find it difficult to compile Linux kernel. Compiling kernel needs to understand few things and then just type couple of commands. This step by step howto covers compiling Linux kernel version 2.6.xx under Debian GNU Linux. However, instructions remains the same for any other distribution except for apt-get command.


Step # 1 Get Latest Linux kernel code

Visit http://kernel.org/ and download the latest source code. File name would be linux-x.y.z.tar.bz, where x.y.z is actual version number. For example file
linux-2.6.27.tar.bz represents 2.6.27 kernel version. Use wget command to download kernel source code:
$ cd /tmp
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-x.y.z.tar.bz


Note: Replace x.y.z with actual version number.



Step # 2 Extract tar (.tar.bz) file

Type the following command:
# tar -xzvf linux-2.6.27.tar.bz -C /usr/src
# cd /usr/src



Step # 3 Configure kernel

Before you configure kernel make sure you have development tools (gcc compilers and related tools) are installed on your system. If gcc compiler and tools are not installed then use apt-get command under Debian Linux to install development tools.
# apt-get install gcc


Now you can start kernel configuration by typing any one of the command:

* $ make menuconfig - Text based color menus, radiolists & dialogs. This option also useful on remote server if you wanna compile kernel remotely.
* $ make xconfig - X windows (Qt) based configuration tool, works best under KDE desktop
* $ make gconfig - X windows (Gtk) based configuration tool, works best under Gnome Dekstop.


For example make menuconfig command launches following screen:
$ make menuconfig

You have to select different options as per your need. Each configuration option has HELP button associated with it so select help button to get help.



Step # 4 Compile kernel

Start compiling to create a compressed kernel image, enter:
$ make
Start compiling to kernel modules:
$ make modules


Step # 5 Install kernel modules

Install kernel modules (become a root user, use su command):
$ su -
# make modules_install
It will install all the module under /lib/modules/ directory.



Step # 6 Install kernel

So far we have compiled kernel and installed kernel modules. It is time to install kernel itself.
# make install

It will install three files into /boot directory as well as modification to your kernel grub configuration file:

* System.map-2.6.27
* config-2.6.27
* vmlinuz-2.6.27

OR

But if don't want to install the kernel on the same machine as on which it i being compiled then.
# make bzImage
Where bzImage is a files created after step 4 in the current working directory.

It will create the three files into current directory.



Step # 7: Create an initrd image

Type the following command at a shell prompt:
# cd /boot
# mkinitramfs -o initrd.img-2.6.27 2.6.27

If using fedora/redhat type the following command at a shell prompt:
# mkinitrd -o initrd.img-2.6.27 2.6.27

initrd images contains device driver which needed to load rest of the operating system later on. Not all computer requires initrd, but it is safe to create one.



Step # 8 Modify Grub configuration file - /boot/grub/menu.lst

Open file using vi:
# vi /boot/grub/menu.lst

title Debian GNU/Linux, kernel 2.6.27 Default
root (hd0,0)
kernel /boot/vmlinuz-2.6.27 root=/dev/hdb1 ro
initrd /boot/initrd.img-2.6.27
boot

Remember to setup correct root=/dev/hdXX device. Save and close the file. If you think editing and writing all lines by hand is too much for you, try out update-grub command to update the lines for each kernel in /boot/grub/menu.lst file. Just type the command:
# update-grub

Neat. Huh?



Step # 9 : Reboot computer and boot into your new kernel

Just issue reboot command:
# reboot

Thursday, November 20, 2008

Networking Basics

Networking Basics

Basic Networking Funda
The traffic/data leaves one machine, goes towards the Gateway, which is then redirected towards the destination by the Gateway.
* In this process first routing table in the source machine should point towards the correct Gateway.
* The Gateway should have correct route, IPtables and natting rules pointing towards the destination.
* The destination machine should have correct IPtables to accept the data comming from that source.

In general Networking depends on three things.

* Routing
* Nating
* IPtables/Firewalls

Routing

It tells tell the system about where to send a packet for any destination.
Routing is the process of selecting paths in a network along which to send network traffic. Routing is performed for many kinds of networks, including the telephone network, electronic data networks (such as the Internet), and transportation (transport) networks. This article is concerned primarily with routing in electronic data networks using packet switching technology.

In packet switching networks, routing directs forwarding, the transit of logically addressed packets from their source toward their ultimate destination through intermediate nodes; typically hardware devices called routers, bridges, gateways, firewalls, or switches. Ordinary computers with multiple network cards can also forward packets and perform routing, though they are not specialized hardware and may suffer from limited performance. The routing process usually directs forwarding on the basis of routing tables which maintain a record of the routes to various network destinations. Thus constructing routing tables, which are held in the routers' memory, becomes very important for efficient routing. Most routing algorithms use only one network path at a time, but multipath routing techniques enable the use of multiple alternative paths.

Routing, in a more narrow sense of the term, is often contrasted with bridging in its assumption that network addresses are structured and that similar addresses imply proximity within the network. Because structured addresses allow a single routing table entry to represent the route to a group of devices, structured addressing (routing, in the narrow sense) outperforms unstructured addressing (bridging) in large networks, and has become the dominant form of addressing on the Internet, though bridging is still widely used within localized environments.

To explore it more you should stop reading and take a look at the routing table in you system
* route -n
This command will give you a table ex.

Kernel IP routing table.




Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.20.0 192.168.20.1 255.255.255.0 UG 0 0 0 tap0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.1.5 0.0.0.0 UG 0 0 0 eth0


To add a route to this table type
* route add -net 192.168.1.0/24 gw 192.168.1.1

You can replace 192.168.1.0/24 with network ip to which you want to send data and 192.168.1.1 with the gateway through which that network is connected.

To delete a route to this table type
* route del -net 192.168.1.0/24 gw 192.168.1.1



Natting

In computer networking, network address translation (NAT) is the process of modifying network address information in datagram packet headers while in transit across a traffic routing device for the purpose of remapping a given address space into another.

Most often today, NAT is used in conjunction with network masquerading (or IP masquerading) which is a technique that hides an entire address space, usually consisting of private network addresses , behind a single IP address in another, often public address space. This mechanism is implemented in a routing device that uses stateful translation tables to map the "hidden" addresses into a single address and then rewrites the outgoing Internet Protocol (IP) packets on exit so that they appear to originate from the router. In the reverse communications path, responses are mapped back to the originating IP address using the rules ("state") stored in the translation tables. The translation table rules established in this fashion are flushed after a short period without new traffic refreshing their state.

As described, the method only allows transit traffic through the router when it is originating in the masqueraded network, since this establishes the translation tables. However, most NAT devices today allow the network administrator to configure translation tables entries for permanent use. This feature is often referred to as "static NAT" or port forwarding and allows traffic originating in the 'outside' network to reach designated hosts in the masqueraded network.

Because of the popularity of this technique, see below, the term NAT has become virtually synonymous with the method of IP masquerading.

Network address translation has serious consequences (see below, Drawbacks & Benefits) on the quality of Internet connectivity and requires careful attention to the details of its implementation. As a result, many methods have been devised to alleviate the issues encountered. See article on NAT traversal.

Nating can be taken care off in 2 ways one is through the router and the other way is if you are dealing with Linux machines it can be done using IPtables.

To check nating tables on a Linux machine type
* iptables -nvL -t nat
This command will give you a table ex.

Chain PREROUTING (policy ACCEPT 3791 packets, 359K bytes)

pkts bytes target prot opt in out source destination


Chain POSTROUTING (policy ACCEPT 6573 packets, 406K bytes)

pkts bytes target prot opt in out source destination


Chain OUTPUT (policy ACCEPT 6405 packets, 394K bytes)

pkts bytes target prot opt in out source destination




NAT or DNAT

Address translation occurs before routing. Facilitates the transformation of the destination IP address to be compatible with the firewall's routing table. Used with NAT of the destination IP address, also known as destination NAT or DNAT


SNAT

Address translation occurs after routing. This implies that there was no need to modify the destination IP address of the packet as in pre-routing. Used with NAT of the source IP address using either one-to-one or many-to-one NAT. This is known as source NAT, or SNAT



PACKET FLOW UNDER NAT

* The packet is first examined by rules in the PREROUTING chain, if any. It is then inspected by the rules in the nat table's PREROUTING chain to see whether the packet requires DNAT. It is then routed.

*If the packet is destined for a protected network, then it is filtered by the rules in the FORWARD chain of the filter table and, if necessary, the packet undergoes SNAT in the POSTROUTING chain before arriving to the network.



Masquerading (Many to One NAT)


Masquerading is another name for what many call many to one NAT. Traffic from all devices on one or more protected networks will appear as if it originated from a single IP address on the Internet side of the firewall.

Note: The masquerade IP address always defaults to the IP address of the firewall's main interface. The advantage of this is that you never have to specify the NAT IP address.

You can configure many to one NAT to an IP alias, using the POSTROUTING and not the MASQUERADE statement. An example of this can be seen in the static NAT section that follows.


Port Forwarding Type NAT


In many cases home users may get a single DHCP public IP address from their ISPs. If a Linux firewall is also your interface to the Internet and you want to host a Web site on one of the NAT protected home servers, then you will have to use port forwarding

Port forwarding is handled by the PREROUTING chain of the nat table

examples:

* iptables -t nat -A PREROUTING -p tcp -i eth0 -d $external_ip --dport 80 -j DNAT --to 192.168.1.200:8080


This rule allows the port forwarding for traffic destined to port 80 of the firewall's IP address to be forwarded to port 8080 on server 192.168.1.200


SNAT
----

SNAT is used to NAT all other outbound connections initiated from the protected network to appear to come from single IP address.


POSTROUTING 1 to 1
------------------

* iptables -t nat -A POSTROUTING -s 192.168.1.100 -o eth0 -j SNAT --to-source 97.158.253.26

Many to 1
---------

* iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j SNAT -o eth0 --to-source 97.158.253.29



* DNAT ----- Used to do destination network address translation. ie. rewriting the destination IP address of the packet

* SNAT ----- Used to do source network address translation rewriting the source IP address of the packet The source IP address is user defined

* MASQUERADE Used to do Source Network Address Translation.By default the source IP address is the same as that used by the firewall's interface


NOTE:

ipforwarding should be enabled..then only your rules will work

cat /proc/sys/net/ipv4/ip_forward

This should give 1.



IPtables/Firewalls

iptables is a user space application program that allows a system administrator to configure the tables provided by Xtables (which in turn uses Netfilter) and the chains and rules it stores. Because iptables requires elevated privileges to operate, it must be executed by user root, otherwise it fails to function. On most Linux systems, iptables is installed as /usr/sbin/iptables and documented in its man page [1], which can be opened using "man iptables" when installed. iptables is also commonly used to inclusively refer to the kernel-level component Xtables that does the actual table traversal and provides an API for kernel-level extensions.

iptables works with Linux kernels 2.4 and 2.6. Older Linux kernels use ipchains (Linux 2.2) and ipfwadm (Linux 2.0).


The source of the packet determines which chain it traverses initially. There are three predefined chains (INPUT, OUTPUT, and FORWARD) in the "filter" table. Predefined chains have a policy, for example DROP, which is applied to the packet if it reaches the end of the chain. The system administrator can create as many other chains as desired. These chains have no policy; if a packet reaches the end of the chain it is returned to the chain which called it. A chain may be empty.

Each rule in a chain contains the specification of which packets it matches. It may also contain a target. As a packet traverses a chain, each rule in turn examines it. If a rule does not match the packet, the packet is passed to the next rule. If a rule does match the packet, the rule takes the action indicated by the target, which may result in the packet being allowed to continue along the chain or it may not.

The packet continues to traverse the chain until either (1) a rule matches the packet and decides the ultimate fate of the packet (for example by calling one of the ACCEPT or DROP targets); or (2) a rule calls the RETURN target, in which case processing returns to the calling chain; or (3) the end of the chain is reached.

This example shows an already-configured workstation firewall. The command "iptables -L" is executed by user root to display the firewall configuration.

* iptables -nvL


Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all -- localhost.localdomain localhost.localdomain
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
REJECT all -- anywhere anywhere

Chain FORWARD (policy DROP)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination


Before proceeding further one should be aware of the three predefined chains of IPtables.
* INPUT
* OUTPUT
* FORWARD

INPUT
As the name explains this chain comes into picture when any packet is destined towards the system.

OUTPUT
This chain has to be configured when thinking of sending packet to other machines in the network.Usually we don't configure this chain.As it's default policy is ACCEPT.

FORWARD
This chain comes into picture when any packet in the network travels from 1 system 2 the other via the gateway, then FORWARD chain on the gateway has to be configured in such a way that it should accept the packets for forwarding.


Adding rules to IPTABLES
* iptables -I INPUT -s 192.168.20.0/24 -j ACCEPT
* iptables -I FORWARD -s 192.168.20.0/24 -j ACCEPT
* iptables -A FORWARD -s 192.168.20.0/24 --dport 80 -j ACCEPT
* iptables -I FORWARD -d 192.168.20.0/24 -j DROP

(Where I/A are to specify the priority of the ip rule, I is the inserting to the highest priority and A is appending it to the last)

Deleting rules from IPTABLES.
* iptables -D INPUT -s 192.168.20.0/24 -j ACCEPT
* iptables -D FORWARD -s 192.168.20.0/24 -j ACCEPT
* iptables -D FORWARD -s 192.168.20.0/24 --dport 80 -j ACCEPT
* iptables -D FORWARD -d 192.168.20.0/24 -j DROP

(Remember the rule should be in the same format as it was added only change is the -D)

To check if you are connected to your network properly commands like ping. telnet and traceroute will help.
See man pages of these commands for more help.
Use tcpdump to view the flow of packets FOR BETTER RESULTS.


Hope this helps.

Tuesday, November 4, 2008

Network Bridging

TAP (Network Bridging)


In computer networking, TUN and TAP are virtual network kernel drivers. They implement network devices that are supported entirely in software, which is different from ordinary network devices that are backed up by hardware network adapters.

TAP (as in network tap) simulates an Ethernet device and it operates with layer 2 packets such as Ethernet frames. TUN (as in network TUNnel) simulates a network layer device and it operates with layer 3 packets such as IP packets. TAP is used to create a network bridge, while TUN is used with routing.

Packets sent by an operating syst/sbin/modprobe tunem via a TUN/TAP device are delivered to a user-space program that attaches itself to the device. A user-space program may also pass packets into a TUN/TAP device. In this case TUN/TAP device delivers (or "injects") these packets to the operating system network stack thus emulating their reception from an external source.


TUN is mostly used for OpenVPN, VTun.
Where as TAP is used for bridging Virtual Machines to the host machine.

We have one more concept that is BRIDGE an this is used if you want to put a number of Virtual Machines into a 1 single subnet.
Else with tap devices you can not do that.



Configuring Network Bridges.


TAP

To configure tap you need to install uml-utilities, to do this ->

* apt-get install uml-utilities

This will give you tunctl command, which is further used to build tap interface.
To build tap interface run ->

* tunctl

This will give you (Set 'tap0' persistent and owned by uid 0).

You can see this interface with <ifconfig -a> command.
Now when the interface is ready give it an ip, make it up and link it to your Virtual Machine.

* ifconfig tap0 192.168.1.1

The ip which you assign to tap0 should be of different subnet as your host network.

Just remember doing this will not solve your full problem of networking in Virtual Machines, to make it fully up and running it requires routing, iptables and nating(NAT) to be configured properly.
You can expect these things to be covered in some of my later posts.


Bridge

Bridge comes into picture when you have more than 1 Virtual Machine.
It helps to get all the Virtual Machines in 1 single network which is not possible with TAP alone.
With this one can also get the Virtual Machines into the same network as the host.

To configure Bridge you need to install bridge-utils, to do this ->

* apt-get install bridge-utils

This will give you brctl command, which is further used to build tap interface.
Running brctl alone will give a list of option which can be used with this command.
But for now run ->

* brctl addbr br0

This will add a bridging interface named br0.Now add an ip to it.

* ifconfig br0 192.168.1.1

Here comes a time to select the scenario that either you need a different network or the same net work for VM(Virtual Machines).

If you need the network to be same then.

* brctl addif br0 eth0
* brctl addif br0 tap0
* ifconfig eth0 0.0.0.0
* ifconfig tap0 0.0.0.0

This will add eth0 and tap0 interfaces to bridge br0.

And if their is no need of the same network then.

* brctl addif br0 tap0

This will add tap0 interface to bridge br0.
In this case to make network live and working configure routing, iptables and nating(NAT) on the host machine.


That it.
Hope it helps.

Vitualization

Virtualization

To start with Virtualization I can say it is the one of the most helpful technology known to man. The basics which make it a great technology is the feature that user can have more than one machine at the cost of one single machine.
Or for a software guy he can get a number of machines to test his development.

In Virtualization we have 2 different types.
They are ->
* Full Virtualization.
* Para Virtualization.

Full Virtualization, in computer science, is a Virtualization technique used to implement a certain kind of virtual machine environment: one that provides a complete simulation of the underlying hardware. The result is a system in which all software capable of execution on the raw hardware can be run in the virtual machine. In particular, this includes all operating systems. (This is different from other forms of Virtualization – which allow only certain or modified software to run within a virtual machine.)
Some examples of such are VMware/QEMU/KVM.



Para Virtualization, In computing, paravirtualization is a Virtualization technique that presents a software interface to virtual machines that is similar but not identical to that of the underlying hardware.

Paravirtualization may allow the virtual machine monitor (VMM) to be simpler or virtual machines that run on it to achieve performance closer to non-virtualized hardware. However, operating systems must be explicitly ported to run on top of a paravirtualized VMM.

And according to me I personally prefer this due to its feature of not using too much of RAM and other resources, as it uses host's resources and that too when required.
Some examples of such are XEN/OpenVZ.



How to?
In terms of how to do virtualization it can be done through GUI or Command line. I prefer Command line for doing the job.

Full Virtualization.

I will start explaining this hoping that you have QEMU installed.

QEMU

qemu-img create -f raw image.img 5G

This will create a blank image named image.img you can define the size of the image by changing the 5G option to any size.
Now it's time to boot it with a cdrom to give it an Operating System. To do this you need either a CDROM on host machine o an ISO image.
If you have an ISO image then enter this command ->

qemu -cdrom linux/windows.iso -hda image.img -m 600 -boot d

Where linux/windows.iso is your ISO image, image.img is the disk image you just created,-m 600 is the RAM you want to allocate to the Virtual Machine and -boot d is to tell the Virtual Machine to boot from the CDROM.

After the installation is over then run the Virtual Machine with the following command ->

qemu -hda image.img -m 600 -net tap -net nic

Where -net tap tell it to connect to a virtual interface(tap device -- tap0/tap1 etc) on the host and -net nic defines 1 network card present on the Virtual Machine.
And if you don't know how to configure tap device you can refer my next post which will be TAP/TUN.



Para Virtualization.

XEN

Under this I will go on with XEN, to have virtualization with xen you need to have the kernel which supports XEN virtualization.
I now suppose that you have upgraded your kernel to support XEN.

To start with XEN you must be familiar with two terms which we will be using a number of times.
* dom0 -> this refers to the host on which the Virtual Machine or the domU is situated.
* domU -> this refers to the guest machine which we are going to create now.

Now if you have upgraded your kernel of the host machine to support XEN, this means yours dom0 is ready.
And it's time to create domU but before that just reboot your machine to boot into the upgraded kernel.


Creating - domU


File Based Setup of Virtual Disk

* mkdir -p /home/haps/
* dd if=/dev/zero of=/home/haps/diskimage.img bs=1024k count=5000

* dd if=/dev/zero of=/home/haps/swapimage.img bs=1024k count=512


* mkfs.ext3 /home/haps/diskimage.img

* mkswap /home/haps/swapimage.img


* mount -o loop /home/haps/diskimage.img /mnt




Debootstrap New OS onto Virtual Disk

* debootstrap --arch i386 lenny /mnt http://ftp.de.debian.org/debian/

* mv /mnt/lib/tls /mnt/lib/tls.disabled


* cp /etc/apt/sources.list /mnt/etc/apt/

* vi /mnt/etc/apt/sources.list


* cp -a /lib/modules/2.6.16-1-xen-k7/ /mnt/lib/modules/


* cp /etc/resolve.conf /mnt/etc/


* cp /etc/network/interfaces /mnt/etc/network/

* vi /mnt/etc/network/interfaces (and make it look like this)


#To use Specific IP address - edit the /mnt/etc/network/interfaces manually.
#To use DHCP, edit and include the following:
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp

* vi /mnt/etc/hostname


* vi /mnt/etc/fstab
(and make it look like this)

proc /proc proc defaults 0 0
/dev/sda1 / ext3 defaults 0 0
/dev/sda2 none swap sw 0 0



Setup domU Xen Config

* vi /etc/xen/haps (and make it look like this)
kernel = "/boot/vmlinuz-2.6.16-1-xen-k7"
ramdisk = "/boot/initrd.img-2.6.16-1-xen-k7"
memory = 192
name = "haps"
vif = ['bridge=xenbr0']
ip = "ip 192.168.1.5"
gateway = "ip add"
netmask = "255.255.0.0"
root = "/dev/sda1 rw"


* ln -s /etc/xen/haps /etc/xen/auto/haps
//link in the config file so that the Virtal Machine starts on Bootup of Dom0



Run and Update DomU

* xm create haps -c
// CTRL + ] Gets out of the Console



All going well you should see domU booting up. Login as "root" with no password. Execute the following commands to update:

* passwd

* apt-get update
* apt-get upgrade

* apt-get install module-init-tools //for iptables
* apt-get install iptables

* iptables -L


For other commands on xen do .


Thats it.
Hope this helps a lot of people.