*Has anyone noticed a difference in 2.5mm adapters?*
All adapters exhibit differences, and as well, there are differences in different types. It would make sense the hard-bodied type sounds better, it's more direct, less signal travel.
The best adapter though, is no adapter.
*Threw one of the cable jacks into the freezer too, maybe that will help? :?*
Yes. Throw the other adapters in as well, and the headphones, USB charger, cable and whatever other accessories you have, such as the case. It all helps.