You cannot remove the pin. It's used in many parts of the system, including security and telecomm. The best thing to do if you're only using the users to segment your media collection is to simply set all the pins to the same simple sequence (perhaps the default - 1,2,3,4).
You may not have noticed this, but the privacy of your personal media is taken fairly seriously. Others really can't access it no matter how they try to get in - SAMBA, Orbiter, even from the core's terminal. This may be slightly overkill for most situations, but it is a pretty neat feature, and you get it with no additional setup cost.