That was a great find, congratulations!
To remove those caps, I simply take a hair dryer and warm the caps. They'll come right off. I use a method similar to Jonathan's for removing the receiver element. After removing the cap, I cup my left hand and smack the handset handle, with the element pointing toward my palm, against my palm. Usually a couple of hits will make it pop into my hand.
Good luck!
~Dennis