I would suggest you start with reading the Overclocking 101 on the forum http://www.computerforum.com/16346-overclocking-101-oc101.html
It is a nicely written guide and even though a bit dated but will clear you concerns regarding overclocking. Then I would suggest using a processor specific guide, to overclocking for the basic data reference and then trying your hand at it.
Also an after market cooler is recommended, I would not recommend overclocking on the stock HSF.