Both the link that Totallymaxed posted as the first reply in this thread and my post which lists the options, both clearly state the xorg.conf file... but no matter
If X tries to use the EDID data to calculate the DPI (and I say again, please read your Xorg.0.log file, you will see it state the DPI it is using - if it is something substantially lower than 90 then this is the issue) and your screen is reporting the wrong physical size, which is very common, then when X divides the resolution you chose by screen's physical size in inchs to get the number of Dots Per Inch (DPI) it will get the wrong screen density. It then uses this to calculate how many pixels large a font of particular a "point" size needs to be, and thus the font is the wrong size.
If UseEDID is false in both locations in your xorg.conf file, then X is probably assuming a screen size as it has no other information to base it on. You can either switch UseEDID to true and hopefully your screen does report the correct size, or just use the DPI option I gave you and manually set the DPI. 90 is normal, 100 would be slightly bigger. I use 130 so that I can read my screen comfortably from the lounge.
Different OS's use different approaches to calculating this, or simply fix the DPI (like Windows so that there is no consistency or relationship between on screen and real world, or another screen). And obviously, if one system uses an assumption about how big the screen is, then what the fonts look like is entirely dependent on what assumption the program/driver made. My screen is a 46" Samsung LCD but it reports in EDID that the screen size is 160mm x 90mm