A router connects networks together.
A router, in your case, can do two things.
1. You will be able to communicate OFF the internet in your own network (workgroup) in which both computers can share files and a printer.. etc...
2. Each computer can access the INTERNET independent of eachother. One computer is turned off the other can STILL access the internet.
I prefer to use a router for that reason.
Note:
If one of the computers is turned off the workgroup between the two computers stops working until that machine is running again... but the INTERNET connection will work independent of the other computer...