Your case is fine, though for $50 I'd recommend a Coolermaster Centurion 5
PSU's are the power supplies that power all your components. I'd suggest this stable, but budget minded unit:
http://www.newegg.com/Product/Product.aspx?Item=N82E16817171046
I'd also recommend a 4850 or 4870 over the...