Disclaimer: It should be noted upfront that this post mainly concerns cooling with an eye toward overclocking. If you intend to keep all of your components at stock clocks, it's unlikely you'll need to bother with aftermarket cooling of any kind. If your intent is to utilize aftermarket cooling to achieve a near silent PC, please ask in the thread for recommendations.
Cooling your PC comes down to a pretty simple decision: Air or Water. For those inclined to nitpick, there is also:
"Bong" Water Cooling
Mineral Oil Bath
These options are extremely expensive and very time consuming, and all require at least some modification to case and components. Phase Change, TEC, and LN cooling are all sub-ambient cooling methods (that is, they cool the processor to below the ambient temperature of the room) and as such can produce potentially damaging condensation, which requires the application of insulation and other motherboard modification to ensure safe operation.
As you might be able to tell, this leaves air or water cooling (or some combination of the two) as the only really feasible options for most home builders. We'll start with air cooling.
Air cooling is a relatively simple setup. You should start with a solid case for air cooling. Some cases (the Corsair Obsidian 700D/800D for example) are very water cooling oriented, with relatively unimpressive air cooling performance. Cooler Master's HAF series of cases are good air cooling performers, as is the CM690 II (which is also a solid mid-tower for custom water cooling). Lian Li makes some good mid-towers with adequate intake and exhaust setups. Fractal Design's R3 and Arc Midi have room for a very nice air cooling setup. Silverstone's FT02 is a good, but very expensive, option. After choosing your case, component cooling is your next stop:
CPU: There are a boatload of different options for air cooling your CPU. At lower budgets, the Cooler Master Hyper 212+ is a favorite, and the Xigmatek Gaia has recently arrived to solid reviews. For a bit more, the Scythe Mugen 2 or 3 is a good option, as is the Xigmatek Balder (an updated, push/pull compatible version of the well-liked Black Knight cooler). On the high end are the Prolimatech Megahalems and Thermalright Venomous X. If you really want to go for broke with air cooling, the Noctua D14 or Cogage Arrow are your go-to coolers.
GPU: Your graphics card comes with a perfectly acceptable cooler already applied. Depending on the make and model of the card, it might be a relatively staid reference cooler, or a tri-fan non-ref monstrosity that just demands that you overclock. Whatever cooler your card ships with will suffice for the clocks it ships at, unless your case is some kind of heat-trapping, dust laden hellhole. Aftermarket GPU cooling options also exist, though I don't know enough about them to recommend one above another.
Drives: Generally speaking most internal drive bays are placed by the front intake fans to receive cooling. There's generally little need to do more here, unless you have high density internal 3.5' bays and are really loaded up on drives. In such a case a couple of fans ghetto mounted to the side of the cage opposite the intake fans will likely improve airflow enough to maintain safe drive temps.
Motherboard: Most enthusiast motherboards are heavy enough on power phases, with big enough heatsinks that they can be cooled passively by whatever air flows over the motherboard via case fans. Side intake fans, downdraft CPU coolers, and small fans for chipset spot cooling are all good options if your motherboard is overheating.
RAM: If your CPU cooler allows for it, you can purchase RAM with nice tall heatspreaders, which will help more efficiently disperse heat. Corsair and others do sell special fan attachments for RAM spot cooling.
Advantages of air cooling include relatively simple installation (in that only a CPU actually requires you to do more than simply install the component correctly). For a fairly simple method, performance is generally quite solid, especially with newer components. From a price/performance standpoint, it demonstrates solid value.
Disadvantages mainly have to do with air cooling's limitations. While it has improved over the years, there are practical limitations to air cooling that cannot be circumvented. The sheer size of air cooling (tall RAM heatsinks, big passive chipset heatsinks, massive CPU coolers, and double and triple slot GPU coolers) can cause clearance issues if not properly planned for.
So what we've seen so far is that air cooling is the most common, and in some senses the most versatile type of PC cooling available to you. We'll be taking a quick look at water cooling soon.