Not to sure about your rig being a AM2 one, but on mine I have a 939 4600+(stock 2.2ghz, OC peek 3.0ghz, running stable at 2.8ghz).
2.2ghz stock with stock voltage (temps were around 29-31idle, 30-33c).
2.8ghz was great, (temps were around 32-35idle, 36-38c).
3.0ghz was okish but got the odd "hick up" every now and again (temps were around 34-36idle, 37-39c).
I have my chipset and CPU water cooled, I got my rig to about 2.6 with AC64 cooler but even that couldn't hold the temps under 45-50c, so thats the reason for the water cooling, plus it looks cool and gave me some experience with water cooling. To be honest if and when I build my new rig I will not be water cooling it again.
I would only suggest water cooling if you really want to push your OC'in to the next level. Can't see any point in doing it otherwise.