Visual is a huge part of discerning depth for me, and in general making sense of what I'm hearing. If I can see that I'm in a small room my brain will try to interpret whatever I hear in a way that can fit into that small space. David Greisinger had some binaural recordings on his website that he'd taken in a concert hall. I used his method to calibrate my headphones to my ears and then listened to the recordings. They seemed unremarkable until I also stared at the picture of the orchestra as taken from that seat. The effect was amazing. By looking at the picture I could interpret and make sense of the spacial cues in the recording and the sense of space and depth became very apparent. I really felt like I was there.
Really good perception may not be the best thing for listening to recordings. An ability to relax and suspend one's disbelief might be more helpful. Really good perception will just make it all too obvious that you are listening to 2 speakers that are hitting your head from just 2 specific directions, creating a bunch of weird phase and interference patterns that don't often occur in nature, and mixing the acoustics of a recording space with the acoustics of your listening space. This will all shout FAKE to anyone who's perceptual acuity can't relax a little.