Compiling 2.6.x Linux Kernels from Scratch
From PhoenixWing
In my experience compiling Linux kernels from scratch, on various linux platforms, I have needed keep a standard way to compile a kernel to my liking. With older systems running 2.4 kernels, and newer ones running 2.6 kernels, there has been a need to keep notes on the different compiling methods I've used for each version.
The following information is my adaptation of the most common method's of compiling and my own, as well as documented reasons as to why I chose to do things in such a way. I am by no means, an expert in kernel optimization or compiling a kernel so it is at maximum effectiveness or small footprint, however, I am open to comments and suggestions in ways to improve. It is also worth noting that compiling the 2.6 kernels is much more streamlined, or in my words "less cluttered" than that of the 2.4 kernels. Say good-bye to "make dep" and "make bzImage", as well as say hello to the combining of "make modules" and "make modules_install" into one single "make modules_install". The less steps something takes to compile, without any loss of personal configuration to my liking, the happier I tend to be.
I hope the following helps someone in search of "how" and "why".
Kernel-2.6.x
The older Linux 2.4 kernel version is still very rock solid and stable today. However, for systems requiring more modern hardware detection and robust drivers, 2.6 is the way to go. Provided nothing is holding you back from upgrading from a 2.4 kernel to 2.6 one, I'd suggest "go with it". The 2.6 kernel has become very solid, and in many ways, much faster, especially in handling linux threads and SMP.
Linux 2.4 kernels used rely on the /dev filesystem to know what devices are present on the system, the newer 2.6 kernels however, rely on the UDEV package and not /dev. Thus, if you plan are upgrading to the 2.6 kernel, do some research on the upgrade process, especially migrating from /dev (devfs) to UDEV. Since that is beyond the scope of this document, I'll spare commenting further.
Personally, I never compile linux kernels in /usr/src/linux. In this day and age, /usr/src is typically empty for a newly installed distro, but on occasion, some odd program when being installed searches /usr/src/linux for various headers & includes, which it should not be doing, they should be using /usr/include. Thus, to prevent system problems with bad programming, I typically compile kernels in my /home directory. Take a look at Symlinks in the Kernel for more info, specifically, the section stating: "I would suggest that people who compile new kernels should: +NOT do so in /usr/src. Leave whatever kernel (probably only the header files) that the distribution came with there, but don't touch it".
1. cd /home/joeuser
Remember, I start off as a basic 'non-root' user here. Unpack the linux kernel source, and change into the new directory:
2. tar jxvf linux-2.6.xx.xx.tar.bz2 3. cd linux-2.6.xx.xx
Next, assuming I either have a working kernel with config file, or I have previously compiled a kernel and need to update or change it, copy the existing kernel config to the source directory. If you don't have an existing config, skip this step. I would also suggest not mixing minor kernel version .config files, ie: don't attempt to compile a 2.6 kernel by trying to upgrade from a 2.2 or 2.4 config file.
4. cp /boot/config-version .config
If you are upgrading from one 2.6 kernel to another and just copied the old config file to .config, then you can opt to use the new one and view new kernel options using "make oldconfig".
5. make oldconfig
Answer yes, no or "module" to new options and go on your way. You can also run 'menuconfig' afterwards, or in place of, 'oldconfig', for a more graphical representation.
6. make menuconfig
I feel it's worth noting here, that if you're new to kernel compiling, that it's a good idea to experiment and try this at home if you're able. If you are doing this on a linux box at home, cudo's. If not, is this a production server? If so, make plenty of backups, and make sure you have a rescue disk handy for that "last resort".
Look at the whole entire configuration. Note that drivers, such as SCSI and RAID may appear in more than one menu, depending on what is being selected. For optimal performance and kernel/module size, only enable what you need to. If scsi is needed during boot (most servers do), be sure to include it in the kernel rather than modularizing it. Though, you may build an initrd image with the kernel modules so they'll be available during bootup, most systems you won't want to do that as it may produce some speed and overhead. If you run LVM, be sure to enable it in the kernel. Also, if you are expected high bandwidth and are interested in twiddling with bandwidth management, you may want to enable QoS and/or Fair Queueing. Note the example below from my own home server:
- Select Proper Processor speed (Pentium III Xeon)
- Select Symmetric Multi-Processing (SMP, or # of CPU's, in my case, 2)
- Enable High Memory Support
- Move to Networking -> Networking Support and "QoS and/or fair queueing", and enable it.
(allows using CBQ for bandwidth management, now, or in the future):
[*] QoS and/or fair queueing
Packet scheduler clock source (Timer interrupt) --->
--- Queueing/Scheduling
<*> Class Based Queueing (CBQ)
<*> Hierarchical Token Bucket (HTB)
<*> Hierarchical Fair Service Curve (HFSC)
<*> Multi Band Priority Queueing (PRIO)
<*> Random Early Detection (RED)
<*> Stochastic Fairness Queueing (SFQ)
<*> True Link Equalizer (TEQL)
<*> Token Bucket Filter (TBF)
<*> Generic Random Early Detection (GRED)
<*> Differentiated Services marker (DSMARK)
<*> Network emulator (NETEM)
<*> Ingress Qdisc
--- Classification
<*> Elementary classification (BASIC)
<*> Traffic-Control Index (TCINDEX)
<*> Routing decision (ROUTE)
<*> Netfilter mark (FW)
<*> Universal 32bit comparisons w/ hashing (U32)
[*] Performance counters support
[*] Netfilter marks support
<*> IPv4 Resource Reservation Protocol (RSVP)
< > IPv6 Resource Reservation Protocol (RSVP6)
[*] Extended Matches
(32) Stack size
< > Simple packet data comparison
< > Multi byte comparison
< > U32 key
< > Metadata
< > Textsearch
[*] Actions (requires IProute2)
<*> Traffic Policing
<*> Generic actions
[*] Probability support
<*> Redirecting and Mirroring
< > IPtables targets
< > Packet Editing
< > Simple Example (Debug)
[*] Traffic Policing (obsolete)
[*] Incoming device classification
--- Rate estimator
- Exit and save to .config
Next, copy the config file to /boot/ for backup and safe keeping. I have read on various kernel threads and forums that some odd programs may, per chance, look to /boot/config for kernel configuration information. How truthful that is, I've never been able to verify. However, I always feel "better safe than sorry", and am willing to do anything to prevent a kernel panic, for whatever reason.
7. cp .config /boot/config-2.6.XX-custom
Build and compile the kernel and any dependencies:
8. make
At this point, the kernel is compiled and before I set out to install everything, I make backups. I want to be able to boot back into my previous setup if things go terribly wrong, or at least be able to chroot into the system with a boot disk as a very last resort. Start off by backing up the existing modules:
9: cp -pR /lib/modules/2.6.XX /lib/modules/2.6.XX-backup (cp -pR = Preserve permissions and file types recursively)
Next, install the modules. If you have compiled a kernel as the same version that you are running, running the next command will potentially overwrite the existing directory. Be sure to have /lib/modules/2.6.xx (current version) backed up! Note that, in cases such as Fedora where there is /lib/modules/2.6.x-4-fc2 or somesuch, you will have little chance of overwriting, unless you are recompiling using a SRPM and 'rpmbuild'. Bare kernel source will always produce /lib/modules/linux-2.6.xx.x version without extra stuff at the end. One recommendation is to add your own version string to the end of the kernel version, in my case, I build my own custom kernels with '-js', thus I don't overwrite stock modules, or a previous version. Typically, I have never recompiled an existing kernel unless I had a stable, working config that I wanted to add additional features to.
10: make modules_install
Now we're almost done. Copy the kernel over to the /boot directory, as well as the symbol library. Why copy the symbol library (System.map) ? There's been a few debates on this, but I've had little to no kernel panics, and I'm the type to "be safe than sorry"... thus, in case some program uses it, I like to be sure it's there. The best documentation on why use /boot/System.map that I've found, and potentially agree with, can be found at Peter's Page: System.map. Also, be sure nothing is writeable or executable by anyone other than the 'root' user. The kernel should be read only by the 'root' user only as well.
11: cp ./System.map /boot/System.map-2.6.xx-custom 12: cp ./arch/i386/boot/bzImage /boot/vmlinuz-2.6.xx-custom 13: cd /boot 14: chmod 644 System.map-2.6.xx-custom 15: chmod 400 vmlinuz-2.6.xx-custom
Most systems rely on /boot/vmlinuz and /boot/System.map and won't look for /boot/kernel-2.6.xx or /boot/System.map-2.6.xx-custom, so we make symlinks to them. With modern Linux based distro's, the reliance on /boot/vmlinuz named kernels is rare. LILO and GRUB both allow for kernels named in any way, as long as your modules match the kernel version, everything typically works. My installations of LILO always refer to the full named kernel image, ie: vmlinuz-2.6.13.1-js and not vmlinuz, but I have yet to not include a symlink to the kernel I boot with.
BE SURE THESE ARE SYMLINKS BEFORE REMOVING THEM! Do not erase real files ... if they're real, rename them to something useful for backup reasons:
- mv vmlinuz vmlinuz-2.6.xx-old - mv System.map System.map-2.6.xx-old - mv config config-2.6.xx-old
Remove any existing symlinks, and recreate them.
16: rm -f vmlinuz System.map config 17: ln -s vmlinuz-2.6.xx-custom vmlinuz 18: ln -s System.map-2.6.xx-custom System.map 19: ln -s config-2.6.xx-custom config
Now, edit the boot loader. This may differ from box to box, but you basically should follow suit with any existing entries. The GRUB config file is normally located at /boot/grub/grub.conf, and the LILO config file is typically at /etc/lilo.conf.
20: vi /etc/lilo.conf
Your entry should look similar to the following:
image=/boot/vmlinuz-2.6.15.1-custom label=custom-2.6.15.1 read-only root=/dev/sda2 rename previous label, for backup reasons: label=2.6.13-old
If you're LILO, or Grub includes a initrd image, you may need to create one, especially if you compiled certain features as modules and will need them to boot. One instance is a root partition formatted using ext3. 2.6 kernels are more known to run initrd images than 2.4, so make sure you have one if you need it.
initrd=initrd-2.6.xx-custom.gz
Create an initrd image, if necessary, using:
21: mkinitrd -c -k 2.6.xx-custom -m jbd:ext3 \
-o /boot/initrd-2.6.xx-custom.gz -f ext3 -r /dev/sda2
This tells 'mkinitrd' to create (-c) a new image using kernel (-k) 2.6.xx-custom (should be the same name as /lib/modules/2.6.xx-custom) using modules (-m) jbd & ext3 (jbd is needed for ext3 journalling) and save to output file /boot/initrd-2.6.xx-custom.gz. The '-f' option tells initrd that the root (-r) device is using the ext3 filesystem.
Next, save grub.conf (no rerunning or reinstalling should be necessary) or if you have lilo, run the lilo command:
22: /sbin/lilo
You should see an entry similar to:
Added: 2.6.15-custom * Added: 2.6.13-old
The "*" symbol means LILO will boot that kernel image on boot. You are now done... reboot, and test to see if it works. If not, use the older kernel and boot back into the old setup until you work out any bugs.
25: reboot
