Your understanding of 'tearing' is correct, I understood what you meant in your description of jitter from the outset...
I haven't an explanation or solution for you, rather an expansion on why I asked which sources you saw this behaviour. In many media files, you will see the exact behaviour you are describing - on LMCE or on a general purpose computer. How pronounced the effect is on a specific platform with a specific media file can vary. But the point is, this effect can easily be an artifact of compression, variation of frame rates, etc. It will often be erratic, sometimes happening a couple of times per second, then going for several seconds before you get another 'jerk' in the screen motion. If the source is the file, then there is little you can do about it as the effect is essentially "recorded into" the file.
Now to define what we mean by a media file. Typically, something like an .avi, .wmv, .ogg, .mkv, etc of the type you might typically downloaded from the Internet. It is quiet common for this to happen with this type of source.... you get used to it. However, if the source is a genuine DVD, that is more troubling. Its not that DVD's cannot experience the exact same artifacts, its just that professionally produced ones usually don't except in the most demanding of scenes. The same goes for a DVD ripped by LMCE - the response will be identical to the DVD original. Any other method of ripping a DVD the question of recompression or transcoding has to be asked. If the result is a smaller file than the original then some form of processing has gone on, and all bets are off, it might as well be a "media file" you downloaded from the Internet for all you can say about whether the effect is encoded in the file, or some issue to do with your system.
Hopefully this info will help you make comparison's - but try not to get too hung up on it unless it is really pronounced. Mild effects like this happen all the time, even on boradcast Digital TV, its when you really focus on them that it becomes noticable and annoying.
Also, de-interlacing settings can cause jittering due to the load, particularly if not done in hardware. Try turning off de-interlace if you have it on, and see if that changes anything.
Your graphics card is already massive overkill, so don't bother upgrading that!
Finally, the only other comment I would make is, if the frame rate of the media is different from the refresh rate that your screen mode is configured at, then inevitably you will see some jerkiness in pan/zoom situations as the frames will not match up 1:1, with vsync'ing this will mean eventually one frame will fall over the edge, so to speak, and miss a refresh on the screen making the motion appear to jerk. Again, not much you can do about that ... its the nature of the beast that media sources are all sorts of frame rates, so its bound to happen at some point.
