Is there actually a difference?

 Lots of good answers above. Yes many big dollar amps are more about hefty power supplies so they can drive difficult loads, as was said previously. My own approach is that if a speaker is a difficult load I don't want it.  Money believe is multi way difficult to drive speakers use a tiny bit of fine detail.  Speaking purely  of dynamic cone speakers. Panel speakers are another story, I can understand why they are difficult loads. 
And as well as alluded to above, the best sounding amps are lower powered simple amps, often times tube based. 
 That being said, I love running a big solid-state amp into a pair of subs for anything below 50 Hz. Off of a separate Output on the preamp. 
 But Audio is highly subjective and I'll never fault anyone for a different path to their own happiness. Just don't buy into all the nonsense that's floating around. 

"Speakers sound different - that is very obvious. But I’ve never experienced a drastic change between amps. Disclaimer that I’ve
never personally ABX tested any extremely high end gear."

In general, on an absolute basis it's easier to hear differences in speakers because their design physically couples and interacts with the room.  Differences in amps can be more subtle but still crucial, and they can also vary by system.  In my system for example, my speakers are relatively efficient and I tend to hear much less difference between amps than preamps.  With more demanding speakers I think differences between amps can become more apparent and meaningful.  But even in my system, subtle differences in amps can be extremely important at a certain level.  "At a certain level" is the critical point here, and this gets to my next point. 

"One step further - even IF we actually can tell the difference and we can distinguish a certain amp 7/10 times under extreme scrutiny, is it really worth the thousands you are shelling out to get that nearly-imperceivable .01% increase in performance?"

First, it's impossible to quantify differences between two components, so let's just call them small perceived differences on an absolute level.  Once you become familiar with truly reference quality sound reproduction and your own personal tastes in what moves you sonically, these "small" perceived differences can literally make all the difference.  The difference between a system that objectively sounds really really good and one that drops your jaw to the ground and/or has the ability to reduce you to tears with the right music could easily be a matter of changing from one amplifier to another.  Given that level of impact, how much extra would YOU pay for that amplifier?  As I said before, "at a certain level" (and level applies as much to the audiophile as it does the system) relatively small absolute perceived differences between amplifiers can ultimately be huge in the context of a good system.  By the way, this doesn't apply to only megabuck systems -- it also very much holds for high-quality affordable systems. 

I'll end with this -- "at a certain level" absolutely EVERYTHING in a system (and the room it's in) matters and can become crucial. 

SS amplification (pre and power) reached such a high level of development over the past three decades that competing units are functionally and sonically equivalent. 

I agree with others who have said "it depends on the speaker".

In my travels, I stopped at a dealer and auditioned a Magico floor standing speaker at first driven by a solid state power amp with a very high damping factor; it seemed in that setup the Magicos needed to be driven pretty hard to "come to life".  When the other sound room became available, the salesman moved the Magicos and played them with a powerful tube amp and to me, they sounded significantly more "musical", had better bass and didn't sound so "in your face".

As I've moved up the line with my equipment, it seems that it's the "synergy" of all the components in the chain that seems most important.