I encountered a problem today when backing up webgui from a samba server to an NTFS volume. Because the uploads folder over time will accumulate folder names like "cC", "cc", and "Cc", it creates an issue when copying the uploads folder to a case-insensitive file system like NTFS (or possibly HFS+ on Mac OS X).
In the case of windows XP, copying "cc" to a folder when "Cc" is already there thinks that you may want to replace the folder...
I am guessing this could also cause severe problems when hosting WebGUI on windows or Mac OS X.
A possible solution is to encode the object id when it is passed in to access the uploads folder (or building the URL to an uploaded image). This would avoid the need to change policy on the format of Object IDs, and it would be simple to convert an existing uploads folder to the new format.
When hosting on a platform with a case insensitive filesystem, it would still be rare to cause a problem. You lose a about a sixth of the precision on the hash, so conflicts would be more likely but probably still suitably rare.
The best way would probably be, as you said, to use a different encoding for the file names while leaving the of the ids alone. Base32 and hex would both be suitable encoding formats.
Another thought. If you are going to work on the uploads folder, why not build it so there is no "folder" per se but a database file or some type of archive like zip? In effect you could "virtualize" the uploads folder so it could exist in a database, zip, folder, ram, etc, depending on the application. I guess this would break the optimizations that a web proxy could perform, for example caching static content like jpg. Also there is perhaps more risk of data loss should the archive get corrupted. I can't say if these are significant drawbacks or not.
You would probably have to design a class interface for file I/O operations like read/write etc (if it doesn't exist already) and also go and update all code that is read/writing the uploads folder, so maybe this would be too much. Anyway just thought I would throw it out there.