Every audio component have an intrinsic noise. The level of this noise depends on the quality of the equipment (high-end amplifiers and speakers will produce unperceptible noise).
But that noise could come from your audio card too. The resulting noise would be smaller if you set the audio card's volume to the max and then reducing the volume from the speakers volume knob.
Also try muting out all the input channels that you're not using (especially the Line-in).