Oregon,
Two suggestions:
(1) Jazz Record Center, 236 W. 26th St. Note: there's no external signage for this store: you have to go to the street address, press the buzzer for their suite, and ride the elevator to the 8th Fl. First time I went here, I expected it to be a hole in the wall given its relatively unmarked location. I was overwhelmingly positively surprised upon entering. Be prepared to spend significant time and money there if you like jazz! Large selection, a mix of new and used. One of my three favorite record stores in the country (the other two being Jazz Record Mart in Chicago and Jerry's Records in Pittsburgh).
(2) Bleecker St Records (note: ***** it's no longer actually on Bleecker St.,**** it's now at 188 W. 4th St. If you're googling and find an address for them on Bleecker St, be aware that that's the old address. Just wanted to point this out, since the location change but same name might be very confusing as an out of tower).
Great selection, many genres, I always managed to pick up at least 2-3 nice records there each time I go.