What exactly is colored sound?

When the original audiophiles in the 1950's coined the term "coloration", they used it in the same way video technicians do now with video projectors and monitors---in terms of the accurate reproduction of color "temperature". In video, that means greens look green, reds red, etc. In sound, that means a violin sounds like a violin, not like a cello. An alto singer sounds alto, not tenor. No "color" (deviations from flat frequency response) added to or subtracted from the recording. But as the op said, how does one know what a recording "should" sound like?

Gordon Holt, creator of Stereophile in the early 1960's, and the reviewer credited, even by Harry Pearson, as the man responsible for creating, or at least codifying, the audiophile vocabulary, was very serious about lack of coloration, considering it the number one priority in the reproduction of music.

Gordon recommended everyone make their own recordings, so as to have a known reference in terms of evaluating a components lack of coloration. I did just that, recording a Jump/Blues Swing Band live, using just a pair of high-quality condenser mics straight into a Revox A77 reel-to-reel. One can never be completely certain what any given commercial recording should sound like; one made by yourself is a different story.

Record friends talking, or an a cappella group live; we are very familiar with voices, and immediately hear coloration in the reproduction of them. Record an upright bass; you immediately hear any "thickening" a speaker adds to it's reproduction. Another instrument very revealing of added coloration is a piano; that instrument has a very wide frequency response, and reproducing the two strings of every key equally in tonality, attack and decay, etc., is still far beyond the capability of any system to do perfectly.