to all above, what nic's do you all use?
If it's forcedeth, you'll have problems, especially with eth1, eth0 just get's a new IP from the router, so it works.
eth1 is given a static ip of 192.168.80.1 and will not work when switching cables.
Look at your ifconfig and research, i suspect your MAC numbers of your NIC's change each time you reboot.
You can try to fix this with 70-persistent-net-rules or the easier way is too just disable 1 nic and put in another gigabit nic that works.
Hope it helps,
Richard