It depends what you're upgrading from.
I bought the same processor, installed it yesterday, and saw about 1100 points difference in 3dMark06 scores from my old 4600+. I must say, there isn't some huge difference, but games like Oblivion (CPU-heavy compared to most) run smoother. Instead of split-second pauses and clips while things load, there are now quick slow-downs and FPS drops. So overall, it's more consistent and less choppy...but not by a whole lot.
That being said, I'm satisfied with my purchase. My BIOS is too outdated and my computer doesn't even recognize what the CPU is, but there is a performance increase nonetheless.
If you're upgrading from a 4600+ or so, expect a modest gain...so it depends on your budget to determine if it's worth it. If you're upgrading from a lower model, then I'd say it's worth it for sure (unless you have the dough to upgrade the motherboard and convert to C2D instead).
P.S. My mobo is weird and didn't include the normal CPU retention bracket for AM2 sockets. So I had to stick with the crappy, stock slab of metal as my heatsink. I ordered a new bracket and will put in the better, copper heatsink and fan supplied by AMD with this purchase, and I'll probably see a temperature drop of 5-10 degrees at least. Maybe that'll translate to more performance?