With speakers, there is a sense of 3-dimentional space with a stage in front of the listener. The listening room has a huge impact on the sound as it provides both reinforcement and cancellation.
Headphones' perspective is all in the listener's head and not out front. One can get closer to the sound of the master tape because of the detail that higher end headphones provide. Room colorations are non existent, so room modes, slap echo, etc are not present to ruin the sound.
At the RMAF I listened extensively to headphones using high end headphone amps and DSD recordings. I was gob-smacked at the sound of the higher end headphones. For example, the Abyss 'phones were so incredible and detailed that I was brought to tears on a Stevie Wonder cut. Stand-up bass and cello was a revelation with the room removed from the equation.
Which is better for reference? Probably headphone because of room issues, neutrality and ultimate detail. Which are more fun to listen to? Probably speakers because of their perspective and staging they provide - and headphone are uncomfortable to wear.
In the end you can't have it both ways.
