You should get yourself a copy of the shop manual, it details these things, only costs you about 14 bucks on eBay for a PDF download.
You need to take out the console to remove the ash tray (did it last year to fix my ash tray cover that wouldn't stay open).
You need to loosen that shifter cable 'tension adjuster' so that you can slide it out from the bracket (the manual says to mark the nut positions so that you can return them to the correct position).
The end of the cable 'pops off' the shifter assembly (takes a bit of force, and I worried I might break it).

 

The bad news is you must remove the whole console to get the ashtray and cover out. Search "remove ashtray" on this forum, there are a couple of threads on removing it.

 

Thanks I just now followed your suggestion and bought a copy of the shop manual, I didn't realize "normal" people could even get it, especially not that cheaply. Awsome! Thanks.

 

Not sure I would call XK owners 'Normal'

Enlightened perhaps,picky maybe

 

Thanks to everyone's help here. The car is now clean, back together and doesn't smell like a Starbucks anymore :-)
What a pain in the butt it was though! I especially can't believe the **** you have to go through, just to remove the ashtray!