I can second the Princeton Record Exchange. The prices and quality are very good. Only a couple hours drive from NY. When I lived in and near New York, I never found a record shop in the city I liked. There are also a number of Web sites which offer new and used records. They are:
Acoustic Sounds www.acousticsounds.com
Red Trumpet www.redtrumpet.com
Elusive Disc www.elusivedisc.com
Audiophile International www.audiophileusa.com
Music Direct www.amusicdirect.com
I have purchased from each of these and have received excellent service from them all.