The Harbeths sound great at low volumes. If you liked them, I wouldn’t look too much further. In general, you might focus on those British monitors - Harbeth P3, Spendor A1, Falcon LS3/5A, etc. They were made for the type of set up you have, and they do voices extremely well.
As an aside, I have owned both the Harbeth P3esr and the P3 Anniversary Model (same/similar to the XD), and I preferred the standard P3esr - a little warmer and more relaxing. Some might argue that the XD is more accurate, but I felt that the standard P3esr really connected me to the music better.
Have fun with your search!