Annecy (quite close to Geneva) is a picturesque and charming French town; for "hanging out" purposes I would rate it far above any place in Switzerland. Besancon is also fairly close, and it has some interesting horological history.
If you make a tour along the Lac Leman (at one end of which is Geneva), you can find great stuff: on one side is Lausanne, then Montreux and the Chateau de Chillon. Lausanne is a pleasant but fairly modern town; from Montreux you can take a mountain train up to the top of something high. There are a few very interesting old towns past the end of the lake; one of them is named Sion, but I don't remember the others. Also from the end of the lake you can get up to Chamonix, which is not itself that interesting but affords you access to excellent skiing or hiking depending on the season, with spectacular funicular rides. The other side of the lake has small towns like Thonon and Evian, which are picturesque but touristy. Many of these spots are covered by train; there is also a lot of available ferry boat transportation from town to town.
The above will not help you see the watch industry, but it may save your marriage/relationship with kids.
Also, in Geneva don't forget to visit the excellent art museums and (if it's of interest to you) the museum of old musical instruments.
If you make your way to La Chaux-de-Fonds for the MIH, it's probably worth continuing to Le Locle. There, there is another watch museum called something like Chateau des Monts that I found interesting in part for its watches (although they're not as good as the ones at MIH) and in part for its location in an interesting old building on top of a hill. (If you're walking from the train station, be ready for a fairly stiff climb.)
Be sure to verify that museums are open before you go out of your way to visit them. There is some sort of municipal watch museum in Geneva that seems to have been undergoing a very long renovation project; similarly with the IWC museum in Schaffhausen, although I am pretty sure that one is open again.