How are you controlling the volume then?
I find the headfi provides some advantages over a normal hifi system.
The relative amount of power needed to drive cans (even something as difficult as a pair of Hifiman HE6) is a mere pittance compared to a regular pair of speakers so you don't have to pay for more power. Just a small amount of high quality ones.
Two, hifi systems are affected a lot by the room the system is in. You can do room EQ or use copious amounts of room treatment but there are limits to what can be achieved. On a good set of cans, the room is out of the equation, as the sound goes from the transducers to your ears directly. No reflected sound, no diffused sound. You can often pick up far more detail in a good set of cans that is hard to discern even on speaker systems that are multiples more in cost.
The downsides to me are in the way soundstage is reproduced. You can experience wider sound stages in headfi with the right gear but fundamentally, the main problem is that the centre of the soundstage is between your ears and inside your head. That can sound unnatural. In a hifi system, the sound is in front of you, with a good amount of depth that mimic a live performance, almost.
There are headphones like the Sennheiser HD800 which angle the drivers somewhat and can create an effect of the soundstage outside your head but it is still a little subtle and doesn't beat even a cheap pair of speakers in creating that deep soundstage.
As for head amps, like with things hifi, it's always best to listen and audition. Some amps have good synergies with some headphones so it's always best to test if the combo works in your system. I found my Schiit Lyr for instance to be a good match for the HD800 and the HE6 but was a tad noisy with my LCD3.
I think Schiit headphone amps are hard to beat for the price and features.
