The act of turning on a computer system and making its operating system to be loaded is called booting. The name comes from an image of the computer pulling itself up from its bootstraps, but the act itself slightly more realistic.
During bootstrapping the computer first loads a small piece of code called the bootstrap loader, which in turn loads and starts the operating system. The bootstrap loader is usually stored in a fixed location on a hard disk or a floppy. The reason for this two step process is that the operating system is big and complicated, but the first piece of code that the computer loads must be very small (a few hundred bytes), to avoid making the hardware unnecessarily complicated.
Different computers do the bootstrapping differently. For PC's, the computer (well, it's BIOS) reads in the first sector (called the boot sector) of a floppy or hard disk. The bootstrap loader is contained withing this sector. It loads the operating system from elsewhere on the disk (or from some other place).
After Linux has been loaded, it initializes the hardware and device drivers, and then runs init(8). init starts other processes to allow users to log in, and do things. The details of this part will be discussed below.
In order to shut down a Linux system, first all processes are told to terminate (this makes them close any files and do other necessary things to keep things tidy), then filesystems and swap areas are unmounted, and finally a message is printed to the console that the power can be turned off. If the proper procedure is not followed, terrible things can and will happen; most importantly, the filesystem buffer cache might not be flushed, which means that all data in it is lost and the filesystem on disk is inconsistent, and therefore possibly unusable.