Why not port Minix to the most common players, those based on the Actions chip. It is a very know architecture well documented on the s1mp3 wiki, there is even an emulator for it. There is some work done for an os called SWAN but I doubt there is much development going on.
So I suggest Minix an os designed for minimalistic embedded systems.( Minix 2 can boot on 4mb of ram and 4 mhz
http://www.linuxdevices.com/news/NS5804062141.html )
As MINIX author Andy Tanenbaum says "MINIX is much smaller than Linux and might well be suitable as the operating system for a watch, camera, or transistor radio." So why not an mp4 player?
The BSD licensing will make it very attractive for other companies to use.
Many developers will be attracted to it as long as someone will be willing to do some preliminary work.
I am aware I can't do this but I hope someone will work on this idea.
The Os will mean that the danger of killing players will disappear, no more annoying bugs and unlimited access to the device's resources.
Something similar has been done for the ipod here using linux :
http://ipodlinux.org/Main_Page