Just had it happen to me
Chances are it's your ISP. I know ours was really bad at one point, causing our old Linksys router to freak out (complete loss of internet connectivity for about 5 minutes). After about a month, they cleared things up and it started working fine again.
Assuming your using a wired connection, there's probably not much you can do. You could try updating any firmware to see if that helps.