Best small, desktop speakers, $500 or under (new or used)

From what I understand, upgrading the crossover makes a marked difference to their sound quality?

Yes- replacing the non-polar electrolytic in the crossover with a film cap, which is easy to do.

The Optimus Pro LX-4 looks interesting too. I'll look at how to mod them.

See if you can find the Pro LX-5- its a better speaker in nearly every way. The only problem you run into with them is that there is rear-firing information coming from the tweeters, so the speakers have to be a bit away from the wall behind them for best results. But they are fast and quite smooth, although at this point the woofers will have to be either replaced (replacements are easy to find) or the foam surrounds on the woofers replaced (foam surround kits are easy to find too).

Type of music preferred, classical, jazz, but rock too. I.e., varied.

Speakers don't have taste in music and no-one has found a way to make a speaker better for a certain genre, so this is no worries.
Now the Rogers LS35/a and all the recent variants (Spendor, Harbeth, etc.) should not be ignored! This speaker is very musical and designed for nearfield listening.

Another vintage speaker that is a lot of fun is the KLH Model Nineteen. This is the smallest KLH of the original series designed by Henry Kloss and has the distinction of being one of the few he did that is not a sealed cabinet. Finding a set in good shape is hard, but this speaker is very fast and cohesive, as it has no crossover and relies on a pair of full-range high efficiency 4" drivers.

None of the speakers I've mentioned so far need any serious power- 5 to 10 watts is plenty!

Hold out for a used pair of Silverline Minuets — you wont be sorry.  And I’d cross the Clairaudient speakers off your list.  They’re awful.  I had the 1+1 on demo and were the worst speakers I’ve ever heard.  Took them to a buddy’s house to hear them in his system and he laughed at how bad they were.  Rolled-off, muddy highs, hollow- sounding mids, and no dynamics whatsoever.  IMO someone at TAS got paid for a positive review because these things are awful.  Did I mention they’re awful?  Anyway, hope this helps, and best of luck in your search. 

Clairaudients are awful...well, that might explain why they can be found for $450 or $550 online.

Would love to find Rogers, Spendor, Harbeth in this price range...a bit of a challenge! And KLH, too!

So, let me let one small confession slip... I presently have some small Pinnacle PN5+ speakers which are a good size and sound ok. I just figured these are pretty mediocre compared to what folks are saying here. And...the company is out of business, too. But maybe there’s a tweak there or I just have something better than I think I do. They were never expensive speakers but...

The Triangle Borea BR02 is $450... they are getting great reviews.  Check them out too.

I have 2 pair of Realistic (Radio Shack) Minimus 7s, they're OK. I used one set for surround rears for years on the shelf behind the couch. I seriously upgraded them to Martin Logan Motion 2s (with AMT) These will work on a desktop 110-25000 Hz. Put a $40 CyberAcoustics under the desk to fill in lower freqs. They are worth the $240/pr while they last. Call Safe and Sound and ask if they have any refurbs ($150) laying around.

The holy grail of desktop speakers are the Monsoon Planar 9s. They are like mini Magnapans. The catch is that the controller connects to the amp\sub via a 9 pin mini din and they are All broken. If you can microsolder an old Mac printer cable would work. I have the pin assignments. Otherwise you can jury rig them to another amp/sub (like I did for my TV) but the SQ will probably be inferior to OEM.
If you are ambitious, Madisound is the winner. I love the pair I built from scratch 25 years ago before they marketed cabinet kits. Probably cost prohibitive by the time you match an amp to them. 
Unless you are dead set on your own amp I recommend powered speakers for better SQ and less clutter. Review the rec's above. Even though I have not heard the Vanatoos I'll wager that they sound the best due to bi-amping (matched to the drivers.)and radiators.  Only a direct showdown could prove me wrong.
If you want to max out your $500 my vote is for Vanatoo Transparent One Encore B-Stock.  You'll probably have to make a request to reserve a pair.