Compiling the Linux Kernel

Seriously, there are many good reasons to do this, but there are also many bad ones. Staying on the bleeding edge is a bad reason, unless you've read the changelog and found out that someone has finally added support for a device that you've been wanting to run under Linux.

Here's an example of a good reason: you're trying to install a commercial product that is provided only as a pre-compiled module -- and it won't run under the latest kernel. Your boss wants you to install the latest distribution, but you have to compile a 2.2 kernel for it. Kinky, but it does happen. Let's get on with it.

It's not all that bad, really. You will have to answer a lot of questions, but the process of giving those answers has been made fairly painless. Those questions will mostly be about the devices inside (or attached to) your computer, because that is what an operating system is all about. The kernel, the heart of the operating system, connects those devices to each other. It enables them to talk to each other. The best reason to make a new kernel -- the only real reason -- is because you have a device that your old kernel can't talk to.

Wouldn't it be nice if your kernel supported all the possible devices? No, not really. That's almost the situation you start out with when you do a clean install of Linux. The vendor or packager of your distribution (RedHat, Mandrake, Debian, Slackware, SuSE, whoever) builds a few kernels that include everything -- the metaphorical IP-enabled kitchen sink. At install time the one that seems to be the closest fit is chosen for you. It probably works, but it also is much bigger and more bloated than a kernel that only has to deal with the exact set of devices you have on your machine. (The situation is even worse in the Microsoft world, of course. They include the kitchen sink... and give it a mink lining.)

Cover Your Assets

This Hands-On How-To is more "how to install a new kernel safely" than it is "how to compile a new kernel." There's a reason for that. You're starting out with a kernel that works, more or less. You want to build a new kernel that works better in some way... but when we install that new kernel, we don't want to mess up your machine to the point that it stops working.

The boot process

When you start your Linux computer, the "boot loader" on the motherboard runs a very small program that has only enough intelligence to find your hard drive and turn control over to the first thing it finds there, the "master boot record." The MBR can hold a smarter program, one that has multiple options, but it's still pretty limited compared to a real OS kernel. For the purposes of this Hands-On How-To, we're going to stick with the old workhorse, LILO, the LInux LOader.

(If you installed GRUB instead of LILO, you will need alternate instructions. Do not blindly apply what you read below -- mixing LILO instructions with a GRUB install may make your machine unbootable.)

Take a look at your /boot directory ("ls -alF /boot" is a good command for that). You will probably see a file called vmlinuz (yes, with a "z", not an "x"). It is probably not a real file, but a symbolic link to another file, which in turn has a name something like image-2.2.24. There may be more than one of these files with similar names. One of those is your current kernel. Which one?

To find out, let's look at the plaintext configuration file /etc/lilo.conf -- the commmand more /etc/lilo.conf is a safe way to look at it. There is a block of commands at the top of the file that are common to all the possible ways you might boot up, and then there are one or more "paragraphs" or "stanzas" which each represent one kernel and one set of options for booting with that kernel. If there's only one "stanza," then there's only one kernel installed and it's the only way to start up your copy of Linux (without using a boot floppy or CD, that is). The nice thing about LILO is that you are not limited to one kernel, though. We're looking at your boot information because we want to save your old kernel as a safety net. It's easy to go back to it if your new kernel fails to start your PC.

Look at the first "stanza" (beginning with "image=") and note the name of the kernel -- in our case, it's /boot/vmlinuz, which our directory listing tells us is a symlink pointing to a real file, vmlinuz-2.4.3-20mdk. We are going to do two things: First, we want to make a copy of that real file; second, we want to tell LILO how to refer to that copy, in case our new "vmlinuz" is no good.

Note that your kernel is almost certainly not named vmlinuz-2.4.3-20mdk, unless you happen to have just installed a copy of Mandrake 8.0! Use the name of your kernel file instead.

Now using your favorite editor, copy and paste the first stanza of /etc/lilo.conf, then change the image and the label in the second copy, like so:

This will allow you another boot option at startup time. If you press TAB at the LILO prompt, you will have two choices, "linux" and "previous". See how each label refers to a certain kernel file?

If you were to reboot now, this change would NOT work. Why not? Because we have not installed the modified LILO configuration. My favorite machine-hosing mistake is editing lilo.conf, replacing the kernel image file -- and then forgetting to apply the change to LILO! So let's not make that mistake. Check your work, then run /sbin/lilo as root. You should see the message:

The asterisk in front of "linux" tells you that it is the default and will boot in 5 seconds (see the "timeout" value in lilo.conf? It says 50, but that is 50 tenths of a second) unless you interrupt it to pick the other choice. You do that by pressing TAB and typing in the other keyword, previous; then pressing Enter.

Now that we've come this far, we have a pretty good chance of building a new kernel and rebooting, or at least recovering to our old kernel if the new one doesn't boot. Let's take a little break.

Building the new kernel

To build a new kernel, you need the kernel source files. They don't have to be "new" source files, by the way; your Linux install CDs should include a set of kernel source files for the kernel version you have installed, and you might want to use those to make a custom install of your current kernel version.

Linux kernel version numbers reflect the Linux development process. In order to allow progress while still giving some sense of security, the development teams maintain two main branches of kernel code. The Linux kernel is currently at major version 2. The "stable" branch has an even number in the next position, while the experimental "development" version has an odd number. Thus, 2.4.20 is a very late "stable" kernel, while 2.5.4 would be an early experimental version. Kernel 2.2.20 is one of the last releases of the stable 2.2 tree, and as such is extremely well suited to a production server. Let's use that for our example.

It helps a lot if you've already installed the kernel sources for your distribution, along with the developer's tools such as gcc (the C compiler) and make. If you haven't installed any of these, getting all the little fiddly bits can be difficult. For instance, near the end of the "build" process, there used to be a small assembly-language program which required that you have an "assembler" installed -- but not just any assembler. It had to be a special program that as far as I can tell, was never used by anything else on a Linux system! At this point, we make a hand-waving gesture and assume that you have installed the required files from CD. (That's another reason you might want to build a copy of your existing kernel -- it's a good way to make sure that everything important is installed. If you can't complete the kernel "make" process with the files that came on your original CD, you won't have much better luck downloading only the kernel source.)

Take a look in /usr/src -- you should see a symlink called "linux" which in turn points to a directory such as "linux-2.4.3". That would be excellent.

A "symbolic link" or symlink is just a shortcut, a sort of fake file that points to a real file somewhere else. It comes in handy when you want to refer to "the directory where the current Linux source is", and not be stumped when "current" changes from 2.2.20 to 2.2.22. Remove the old symlink, create a new one, and presto! /usr/src/linux keeps the same meaning -- "here's the current Linux source directory" although it's pointing to a different real directory.

So, we want to remove the symbolic link /usr/src/linux (assuming it IS just a symbolic link!) and re-create it pointing to a new directory, "linux-2.2.20". Here's how to do that:

Note the "arrow" in linux -> linux-2.4.3/ and that the first character in the line is a lower-case l -- that's how you know that "linux" is a link, and not a real directory. You will only see that in a long directory listing, ls -l. If "linux" is a real directory and not a symlink, you'll want to rename it, not remove it. That would look something like this:

Let's go back to the original example now, where we have an existing linux symlink pointing to a linux-2.4.3 directory. (Remember, we are about to compile an older stable kernel, version 2.2.20, to support a commercial module that we can't run under a 2.4 kernel.) Let's begin by removing the /usr/src/linux symlink.

Why do we care about this "linux" symlink so much? The Linux kernel sources are picked up from a "linux" directory when they are packaged into a "tar" archive file; when you unpack them, therefore, they expect to go back into a "linux" directory. If you only have a single "linux" directory, and you unpack them into /usr/src without doing the steps above, they will overwrite some of your existing source files, and leave you with a real mess. You'll have old and new files all mixed together. In a situation like that, the only thing to do is remove the entire "linux" directory and start over. Using a symlink to point to the current version -- and keeping each version in its own directory -- makes it much easier to keep everything under control.

Now you have a nice, empty /usr/src/linux directory. Visit one of the mirrors of ftp.kernel.org and find the source package you want to build:

The actual configuring and building of the kernel is rather an anticlimax. You have a few choices as to how to configure -- "make config" is a very frustrating one, as it has no way to back up if you make a mistake. "make menuconfig" is much more forgiving; it requires that you have the "curses" screen control package but it will work on a text console. "make xconfig" is the obvious choice if you're running X11 (KDE, Gnome, etc). There are help blurbs for each choice; don't be shy about using them. "make oldconfig" assumes you have the configuration from a previous "build" of the kernel -- if you can copy the configuration file from such a build and use it as input to "make oldconfig" then you only have to answer the questions concerning something you want to change, or for new features that did not exist in that prior kernel version.

The main thing to be concerned with is making sure that you build-in support for the devices that you use to boot your computer. If your copy of Linux is installed on an IDE hard drive, you had better build in IDE support. If you are using SCSI hard drives, you need the appropriate SCSI "low-level driver" support. If you aren't sure what devices you have installed, scrolling through the "dmesg" startup screen may help; it tells you what Linux detected last time you booted, so you can be sure that you and your new kernel agree on what to call your devices. Scroll the dmesg screen by typing "dmesg | less".

When you've answered all the questions about what to compile into your new kernel, building it is a yawn. You simply type in "make dep; make clean; make install" and then go have a pizza.

When you come back, we hope to see a message about something called tmpPiggy and a note about how big your new kernel file is. If you don't see that, you're going to have to do some investigating to see what failed. But let's assume it worked. :-) There's a tiny bit more to installing the kernel than moving it to /boot -- you used to be able to get away with that, but these days it's safer to use "make install" so as to pick up the map files and other stuff that the kernel needs. What should happen is that "make install" will replace your existing vmlinuz with the new kernel and run /sbin/lilo for you. You should see the same lines we saw when we ran /sbin/lilo ourselves:

This is important, because 'previous' is our lifeboat if the new kernel doesn't work. If you don't see those two lines above, do NOT shut down or reboot until you have run /sbin/lilo yourself. Otherwise, your only good way out of an endless LILILILILILI... at boot time will involve a boot floppy, or using your install CD as a rescue disk. "man mkbootdisk" is your friend -- don't shut down your system at this point without making a boot floppy. If you do, don't you dare say I didn't warn you!

If you are not sure that "make install" ran the /sbin/lilo command, it doesn't hurt to run it yourself from the command line. If LILO isn't aware of your current kernel, it WILL NOT BOOT successfully. Trust me on this one!

Modules

One of the nicer things about Linux is that you don't have to build everything into the kernel, except the things (like IDE support) that you need in order to boot. Other devices that come and go can get their support in the form of modules. During the config process you tell "make menuconfig" that some things will be built as modules. Once you've run "make install" you can then do "make modules; make modules_install" to build those modules and put them where the new kernel can find them. Note that there are subdirectories under /lib/modules based on your kernel version number -- that's how the module system keeps track of them.

Earlier we used the example of a purchased commercial module that wouldn't work with a 2.4 kernel. When you configure modules, one of the questions is whether to include version information. If you don't do so, you kernel will be suspicious of any modules that were not compiled specifically for that kernel. In general, if you say "yes," any modules from the same major version should work -- for example, you could upgrade from 2.2.18 to 2.2.20, which are both considered 2.2.x kernels, without having to compile all your modules again. The distinction between 2.2.x and 2.4.x is that you are NOT guaranteed that modules for one will work with the other. The "application program interface" or API is allowed to change significantly on a major version change, and your old modules may give you errors saying that they have too many or too few values to work with.

You can discuss this article with the author using the Feedback section of the Brass Cannon webboard.

A Hands-on How-to^SM

from Brass Cannon Consulting

So you want to compile a new kernel.

Cover Your Assets

The boot process

Building the new kernel

Modules

Compiling the Linux Kernel

A Hands-on How-toSM

from Brass Cannon Consulting

So you want to compile a new kernel.

Cover Your Assets

The boot process

Building the new kernel

Modules

A Hands-on How-to^SM