Is this a laptop or desktop? Also, is your sound card integrated to the board? And has this been an ongoing issue or did it just start recently?
Suggestions: boot up with a different mouse and see if the problem persist. And also try to find the latest driver for your sound card (there may be a fix/patch for an issue like that). Another one (wouldn't hurt), try plugging in headphones and see if you still hear it, it may be feed back from an unstable current or old speaker cable.