That's the problem with torrents - you're limited by the upload speed of whoever you're downloading it from. Just because you're downloading it from 10 people doesn't mean it will be fast. If they're all using dial up, or broadband with a low upload speed, you're download will be slow.
Having said that, you may want to make sure that there's nothing in your settings restricts the speed of your downloads (torrent programs will generally allow you to do that). Also, if you're connecting through a router and haven't forwarded the ports, it can result in slow downloads. If you ever get high download speeds, this isn't the problem.
Of course, you can always try a different program, Azureus, for example.
My feeling is that there's probably nothing wrong, it's just the limitations of the people you're downloading the file from.