the pros of overclocking a CPU is, it will run each core faster therefore be able to run more cpu demanding programs. Load and run more programs. Open programs faster. Run high cpu usage games with better fps etc.
The cons are that it is overclocked, therefore the voltage will be higher. This will mean the CPU will run hotter and you would need a better cooler than a stock one to cope with the heat. If not correctly managed then it could easily overheat and cause possible damage to the CPU and other components.
With it overclocked it is running over its factory voltages meaning it may be less reliable depending on voltage increases and may void warranties.
With a gpu its pretty much the same story the pros are that it will run more demanding graphical programs such as games, rendering software etc.
At a higher clock speed it will have a better fps rate than on stock settings.
The cons are that it will get hotter therefore it will need adequate cooling and air movement within the case to give it a constant cold air supply. It is more likely that the gpu will become less reliable over a CPU.
It also reduces the life of components when you overclock them. As far as I know though, this isn't really relevant unless you plan on using your parts for more than 4-5 years.