I think jitter sounds like the jingle of loads of money in audiophile manufacturers pockets for expensive products. The marketeers love the sound of "jittery" and fearful audiophiles who come forth, wallet in hand, with a concern that they have some intractable and immeasurable jitter problem with their system. (immeasurable problems are immeasurably hard and expensive to fix!)
Depending on which "experts" you believe it is either still a problem today or it is something that was caused by poor clock design circuits (or sharing of clocks) in the distant past of digital circuit designs like the early CD.
In theory over sampling is meant to drive jitter problems well outside the audible band where it can be filtered out. Also, accurately clocking things out using a buffer with a dedicated and accurate clock is cheap and is known to minimize jitter.
In theory it can sound pretty much like any other noise or distortion ....as the jitter creates low level frequencies that were not in the original signal. The problem is exacerbatted if jitter is significant and repetitive (as opposed to random) and therefore correlates to a specifc noise signal rather than whiteband noise. This should not normally be the case with most recent designs, even low cost ones.
This is what Bob Katz has to say on what it sounds like;
Here are some audible symptoms of jitter that allow us to determine that one source sounds "better" than another with a reasonable degree of scientific backing:
It is well known that jitter degrades stereo image, separation, depth, ambience, dynamic range.
Therefore, when during a listening comparison, comparing source A versus source B (and both have already been proved to be identical bitwise):
The source which exhibits greater stereo ambience and depth is the "better" one.
The source which exhibits more apparent dynamic range is the "better" one.
The source which is less edgy on the high end (most obvious sonic signature of signal correlated jitter) is the "better" one.
http://www.digido.com/portal/pmodule_id=11/pmdmode=fullscreen/pageadder_page_id=52/