When you switch the PC on, control is passed to a small program on the motherboard called the BIOS, the settings of which you can change, normally by pressing Delete, Esc or some other key while it’s running. The BIOS settings include the definition of the order in which devices should be booted from; for example, the BIOS might try the hard disk first, then if that doesn’t work, USB, then CD, then floppy disk. A device can be booted if its first sector is a boot sector, and as soon as the BIOS finds a boot sector it passes control to it. It in turn determines what should be done next, normally loading the operating system. In normal day-to-day circumstances, the main hard disk is the first device in the BIOS list, this will normally be bootable, so control is passed to it. If however you want to boot a system from a removable device, for example from CD, you can change the BIOS to try the CD drive first. If the CD drive contains a bootable CD-ROM, it will use that; if however it contains an audio CD, this won’t boot so it will try the next device in the list.
In the old days, devices (such as diskettes) were quite small, so there was only room for one operating system on the device. When passed control from the BIOS, the boot sector would simply start booting that system. MS Windows still uses this technique. Nowadays however, devices such as hard disks or USB sticks are much larger, so there is room for several systems if you want. The trick is to load a special program called a boot loader into the boot sector, which decides which operating system to load based on a control file. This control file can either load a default operating system or display a menu where the user can pick which system to load. There are two main hard-disk boot loaders under Linux, called Lilo (LInux LOader) and Grub (GRand Unified Bootloader). The boot loader does not have to load from the device it’s installed on, but can load an operating system from another device, such as CD or diskette. Similarly, a diskette or a CD can simply load the operating system on it, but it can also contain a boot loader, which can load a system from itself or from, say, a hard disk. This is the technique used by the so-called rescue diskettes: if you screw up your hard-disk boot sector so it can no longer be loaded, you can boot from a rescue diskette and fix the problem. See, for example, loadlin (a Windows program for loading Linux), syslinux (for loading Linux from FAT/DOS-formated diskettes), and isolinux (for loading Linux from CD-ROMs).
So, the boot loader passes control on to the selected operating system, which sets up the various hardware it has to deal with and continues with the booting process. If the system is Linux, the boot loader also passes on various parameters, of which the most important are the root filesystem and the initialisation program. Once it’s finished setting up the hardware, Linux mounts the root filesystem, and passes control to the initialisation program.
There are various howtos which give further info on the boot process (see Linux Documentation Project), though they tend to be rather out-of-date.