Shadorne is basically correct. I did indeed have in mind books on music as opposed to something like the Alton Everest book. However, I will say that ear training will only help critical listening to equipment, etc., as this kind of listening should still be based on what you are hearing (or not hearing) in the music, and how well you can or can't hear it. Any formal ear training course at a university music school will include some basic coverage of acoustics, and so do many books aimed at music lovers and/or amateur players. As far as audiophiles go, different folks have different sonic priorities in their systems, and some of us professional musicians find some audiophile's priorities to be very strange indeed.... but to each his own. There is plenty of equipment out there designed for all sorts of tastes.
As to T bone's question on which books would be worthwhile reading, this would depend on what one would want to learn more about. There are many different types of books out there -as Shadorne says, it's a massive subject. I think any audiophile who wants to learn more about how music is constructed, etc. cannot do better than the Aaron Copland book I previously mentioned - What To Listen For In Music. It covers many different topics, and has great suggestions for listening and further study. Learning more about the subjects covered in that book will greatly help any audiophile refine their listening, and therefore their discriminating tastes. There are chapters on the four basic elements of music - rhythm, melody, harmony, and tone color - and other chapters on subjects like texture, structure, forms, the creative process, the relationships between composer/interpreter/listener, etc. It would be a great starting point for anyone wanting to learn more about music of all types, not just classical. There are also alot of different books out there on music appreciation, music history, music theory, etc, far too many to mention here. If anyone has a specific interest that they would like to pursue, feel free to send me an email through audiogon and I would be happy to help where I can.