In terms of "evokes a corrupt town where it’s always raining / paranoia" but doesn’t-sound-like-"general jazz"-but-more-in the-orchestral-realm type thing, I would recommend:
Experiment in Terror (1962 score) - Henry Mancini
Vertigo (1958 score) - Bernard Herrmann (wow! amazing music!)
Sunset Boulevard (1944 score) - Franz Waxman
I’d recommend the Frank Sinatra LPs In the Wee Small Hours, Where Are You?, Only the Lonely, No One Cares, and All Alone.
Taxi Driver (1976 score) - Bernard Herrmann (this one may be a little too close to ’general jazz’ for you, Herrmann composed it with no strings and the main theme is certainly ’jazzy,’ but the greatness of Herrmann is in full effect and the evocative tone you speak of is certainly exemplified here).
Along similar lines is Bill Lee’s score for She’s Gotta Have It. That late-night-in-the-city, dark vibe on songs like "Opening Credits," "Brooklyn Bridge," "A Thought," "Ferrybank Restaurant," "The Hawk." While a bit more major key / upbeat, I really enjoy the song, "Who Will Be the One." Unfortunately, it appears the version that plays over the end credits of the film, is different (IMO, preferred) from the one on the Soundtrack. The sound quality on the soundtrack is superior, but the music has a deeper, more rich emotional quality on the film’s closing-credits version.
As is the case with all the other films I mentioned here, I couldn’t recommend the film more highly!
