Speakers don't do math. Nor are they polymaths given the numbers of instruments, materials, musicians, tuning, styles, etc. etc. in existence.
Yes it's fair to say that all speakers must either be adding or subtracting to timbre
Timbre is the human (expert) perception of the sound of a note made by a specific (tuned) instrument, brought into existence by a musician.
Instrument. Musician. Human Perception of Sound. Note: No speakers involved.
From this point on, there is a very long chain which attempts to provide a 'facsimile' of that note. What you are hearing in your listening chair has to do with that entire chain.