Does it ever bother you

Yes agree with taters.Whenever I suffer through a long tech deep dive I ponder my theory: they've never heard Fela Kuti.

Technical knowledge is very important to be able to maximize the performance of your equipment.  90% of audio performance can be explained and or measured, with the remaining 10% subjective and preference. 

If your question is towards technical jargon with respect to music theory, composition, interpretation used to describe the music (not the sound quality), then I agree with you statement manly because I am not well versed in that area.

It only bothers me if what is said is nonsense.

It always helps when they give a point of reference so you can relate to what they say, but it doesn't "bother" me.  Some people are technical oriented some are music oriented.  If they get excited about technical aspects so be it.  It does "bother" me when people make generalizations about music types not being worthy of high end audio, but that's another discussion.

Some people are technical and some not.   If not tech-speak may as well be a foreign language.  But one worth learning something about if your goal is good sound for reasonable cost.