What's your exact CPU model? Not all CPUs support virtualisation and all virtual machines tend to be somewhat slow without hardware virtualisation support. Also, have you installed the virtualisation extensions (I think that's what they were called anyway) under your guest OS? I think there should be a menu option for that...
Anyway, whether or not RAM is the actual problem depends on what you do on the virtual machine... before upgrading anything, though, I would try increasing the amount of allocated RAM for the virtual machine to, say, 2GB. If that fixes your problem, you really only need to consider getting more memory if you don't have enough left over to do other things - which probably won't happen unless you game and run a virtual machine simultaneously. And if it doesn't fix it - -well, you know that RAM most likely isn't the culprit here.