Basically, at this point, your only saving grace would be multiple Hauppauge HD-PVR capture devices. These capture the component signal, but would essentially solve the distribution hell of doing HDMI switching.
However, since these devices are still relatively new, the driver is still in development. I do not have any of these devices here, so I can't attest to the stability of the current driver in Janne's mercurial repository.
Then a device template needs to be made to control the device.