To be honest it sounds a bit like X is bombing out for some reason - this could be for any number of reasons (including drivers). In addition to Jon's suggestions, check your /var/log/xorg.0.log file - this is the X log file, it is most useful during X start up to troubleshoot when X won't even start. Not sure if it continues logging after it has started, but if it does it may contain more data. Read all the stuff where it is setting up the display device and chipset, and mode validation. But also at the very end of the log file (usually a lot of repetitive lines down there, but go right to the end to see if there is anything interesting.)