For most individuals, the only place inside their computer where they will deal with a thermal compound will be installing the CPU onto a motherboard. In fact, if they have purchased the processor at the retail level with a heatsink, it will likely already have an approved thermal pad already placed on it. Other third party CPU cooling solutions often come with a standard silicone thermal grease to use. For the average user who will not be overclocking their system at all, this is acceptable.
Overclocking processors tends to generate heat that is detrimental to the stability of the computer system. As a result, those individuals looking to overclock their processors should definitely look into high performance thermal grease to use on their CPU cooling solutions. Even a few degrees of difference can determine if the system is stable or crashes. There are concerns to this though. Using non-approved thermal grease on a CPU will often void the warranty from the manufacturer. Of course, if the processor is being overclocked, the warranty has already been voided.
Most average computer users will never have the need to use thermal tape or epoxy. This is generally used in situations where a user wants to add additional cooling elements to components inside the computer system. A typical example would be an individual adding heatsinks to the memory modules that reside on a video card. The video card memory modules do not have any means for mounting the heatsinks to the chips so an adhesive compound is needed. Thermal tape is easy to apply to the modules and the heatsink, but the thermal resistance is very poor. Users typically do this for overclocking the modules on the card, so they tend to go with the thermal epoxies instead. Hope this helps.