You have a device that is not the computer you are using to test that is doing some kind of backup to some random cloud service at night? If you are hooking up every device and its cousin to your new router and the time rolls around where lets say your phone backs up, bam your issues arise?
If you do have a device uploading/backing itself up, you are going to see your connection go to a crawl. Specially when your latency is through the roof. The connection is constantly fact checking all the data you might be backing up, the new traffic has no way to actually go out and request lets say a news page you just clicked on during this event.