How can I tell if I am overdriving the amp or the speakers?

It's called "distortion," and you should turn it down until it doesn't hurt behind your ears--you know. That's how loud you can play it for now.

Next step is make an upgrade to lower the noise. Begin with cables.

Enjoy. It's fun!

@curiousjim  I think that is a serious possibility.

Some nights, nothing I listen to sounds what I would describe as great, and I attribute it to my mood fluctuations or maybe how tired I am.

Also, noting that you typed that the time of day affects what you hear, maybe it is the power coming into your house?  Although I would actually think the power should be "cleaner" in the evening.  Are you running any dedicated lines for your system?

@immatthewj,

I have pretty good power here, when it works. We’ve had seven transformers blow in the last fifteen months! I am a Audience AR6 power conditioner. 
 

Thanks for your thoughts.

JD

A perspective from a non-technical music lover:

I have a two watt tube amp matched with 92 efficient speakers. My room is about 210 sq ft. The sound was flat, and compressed (in terms of sound stage).  In addition, the bass was thin, and frequency response was distorted in the midrange.  I assumed I needed more watts or more efficient speakers.  Hence, I tried speakers that were 98 efficient.  Nevertheless, the subsequent results did not improve. The new speakers played louder but with the same issues.

Through further exploration of my room, I found the ideal speaker placement for my room and then added some diffusion and absorption treatments.  

In short, my 2 watt amp and 92 efficient speakers now perform as originally desired relative to my preferences - warm with natural timbre, sufficient detail,  and excellent sound staging and dynamics.

Know Thy Room and Thy Self

@franklinb66,

“Know Thy Room and Thy Self“

I like that and I have been working on it!

 Thanks.

JD