Overclocking is when you run your CPU's clock at a higher speed than it would normally run at. For example, I have the AMD Athlon 5600+, who's stock speed is 2.8GHz. After buying it, I went into BIOS, tweaked the front side bus and the multiplier, so it would run at ~3.2GHz (which is roughly what an AMD 6400+ would run at, getting more out of my CPU).
Seeing that you are buying a computer that I assume is from a major retailer, overclocking might not work. Some retailers use good CPU, RAM, and Graphics cards, but then make their money by selling you a terrible motherboard, power supply, etc. It takes a pretty good motherboard to get a stable overclock. That as well as the fact that some companies lock the front side bus (FSB) making it almost impossible to overclock. Might I ask if you supply a link to the computer you are thinking of buying? Read more about overclocking
Here.
I can't imagine you noticing a major difference between the 3800+ and the 4400+, seeing as how you are upgrading from a single core 2800+, so at a lower price, and more RAM, I think the 3800+ machine would do just fine.
As for your question about seeing a difference from the 2800+ to the 3800+ X2, I'm sure you would notice an amazing performance spike, because not only does the 3800+ have another core, making multi-tasking quicker and easier, it also runs at a higher speed, and has a newer chip. I hope I answered a lot of your questions, because my fingers are starting to hurt =)
