Definitely risky with your room and amp. Your amp is very nice but has a 120 watt rating at 8 ohms 1Khz. Full bandwidth is more like 70 watts and probably no more than that at lower impedances.
The Harbeth 40s really need authoritative control of their large woofer or the bass will lag and be tubby sounding.
To get the best out of them requires a robust solid state amp stable at low impedances. Not that the Harbeths have low electrical impedance but the 12" woofer demands a solid reign.
Last time I heard them they were driven by Vinnie Rossie L2 signature MOSFET monoblocks and they sounded fantastic.
Time before that they were driven by a Parasound a21 and sounded bloated as heck.