what settings are you using ?
I'm using exchange and i leave my inbox open overnight so that the mail downloads.
otherwise if i come in the morning and i have mails it will take ages to open.
Then again our problem maybe different. does it happens to outlook ? what about other office apps ?
you could also archive some of your folders