Hi about a year ago i made a similar system change like you plan too. Going from a moderate efficient speakers and huge SS monoblocks to High efficient speakers and to single ended tube monoblocks. (80W/8Ohm)
I think its likely easier to find superb sounding poweramps with small wattage than superb sounding mega wattage poweramp.
you are looking at the right path, I am Sorry i didnt do it years ago.
Like many already wrote i too think the ARC Ref5 s.e. is a great preamp but i would strongly recomend to drive it with DR tubes and a good fuse as well as Stillpoints Ultra 6. (Like the rest of the equipment)
I had both the 40th Anniversary and the Ref 10 and they really excelled with these tweaks.
Another isue is weather you will go ballanced or singleended all the way.
My speakers are 100db/1W 8 ohm flat but still like min 80-100Wclass A or min 150AB if i want to shake the house so i would warn about too small amps if you like volumen at live levels.
Stay away from 4 Ohm speaker they stress the amps too much and best if you have min 10 db headroom, this eases the sound pretty much and huge trancients are played with ease.
Therefore IMHO all AN speaker are all too small and so are there poweramps.
Happe lisining