The output impedance of the power amp (more commonly referred to as damping factor)/loudspeaker "modulus" of impedance relationship is one of the most consequential in all of hi-fi (along with loudspeaker/room interaction, pre-amp output impedance to power amp input impedance, and cartridge compliance to tonearm effective mass).
Music Reference's Roger Modjeski encourages owners of his amps to try "light loading" (running the loudspeaker on the impedance tap half the speaker's nominal impedance), but advises against doing the opposite (an 8 ohm speaker on the 4 ohm tap good, a 4 ohm speaker on the 8 ohm tap bad).