I just bought an analogue sound meter today. It looks like my from the needle moving around that lowest levels are around 80db and highest levels peaking around 90db. Average listing levels is probably around 85db.
Is this too loud?
No, not really. That is relative though...my wife thinks it's loud. It's pretty much how loud I enjoy music as well. When it goes to 100db and up that is ear-damaging territory, especially if it's staying in that region for very long.
Now that you have a sound meter you can check how your room might be affecting the sound. You'll have to get a test disc or some other device that generates a fixed frequency spectrum from low to high. Put your sound meter in the sweet spot where you listen...if possible fix it there on a tripod or some other fixed support. Go through the frequency range on your cd player and take note of the volume of each frequency from that point. You will note that there are many dips and peaks. This is entirely normal, but you can look for extremes where variances are more than, say, 6db. If you really want to get technical you can then plot out a graph of the room response and then repeat when you make changes to see how that response graph changes. In this way you might isolate specific problems...for instance, you may find that in order to listen to your midrange at a given volume that you are pushing the highs way louder than that. Or, as was suggested above, that there is a suckout in the vocal region that causes you to push the volume up higher (same idea).
This is just one approach, which does not invalidate the other theories of what might be going on, but it's a good start and very useful information in understanding your room interface (which is VERY much a part of your system).