I've taken a brief look into this as I have XBMC installed here, seems to me there are 2 (minor) issues.
1. XBMC mangles filenames with a colon ':' in them, these are by default used in links to NAS storage for example. No big deal as these can be renamed in the web admin.
2. The only way I could find to set the position of the playing file is in percentages, which I suspect will be way to crude for the "follow me" functionality.
Simple wgets could even be used to control it, for example getting and setting the volume:
# wget -q -O - 'http://xbox/xbmcCmds/xbmcHttp?command=GetVolume'
# wget -q -O - 'http://xbox/xbmcCmds/xbmcHttp?command=SetVolume¶meter=50'
Just in case you're wondering: yes, I have added an entry for "xbox" in my /etc/hosts file.
I think this would make a very nice orbiter-less media director.