Is it fair to say Harbeth is not the best choice for an all arounder and perhaps better with classical or slow jazz?
I couldn’t lock myself into a one genre style speaker. But I love the looks, the wood, the sound they deliver with certain styles of music.
No, it's not. There may be certain types of music where their strengths really shine, but that doesn't mean they are limited to those genres. Get yourself some 40.X Harbets and you can totally rock out. I listen to rock more than anything else and my Super HL5 Plus do just fine with rock.
It's ironic that some folks think certain speakers are great "rock speakers".
Take Klipsch for example. They are super sensitive and people can get them to play really loud, so therefor they are "great for rock". I have (and still do) owned many Klipsch speakers. Played at high volumes (let's say 80db+), I think they are painful to listen to. At moderate volumes, they are really dynamic and can do some special things with vocals and stringed instruments like acoustic guitar.
With the right room, treatment, setup, and amplification, they can sound great, but those stars rarely align and they usually get quite brittle in the highs when pushed hard, which is what it seems most people want to do with them. Not to mention a lot of them aren't constructed that well and without some additional bracing and other measures, you'll hear the cabinets. I love my current Quartets, but I've done some work on them, and they sound best when playing... you guessed it, jazz vocals, acoustic guitars, and the like.