Hmmm....I wonder then why Wavelength, Empirical Audio, Ayre, Audio Research, Locus Design and the majority of the market that tailors in some form to music servers or computers decided to implement USB. Their stuff must sound terrible.
It gets tiring hearing blanket statements about this or that. Most of the higher priced gear, regardless of implementation, is good, some of it very good and opinions always very, even blanket opinions. Boutique companies won't make money for long if their gear is built around poor designs and sounds like crap.
Emre, like many of us who have dived into this end of the hobby, study up on some of the better DACs (Wavelength, Ayre, Audio Research, Red Wine, etc.). Read the reviews and comments from those that own or have tried them. Pick one that you feel may suit your tastes and give it a go. Buy used and and more importantly buy something that would resell relatively quickly should it not suit your taste. You'll be fortunate if you find something that perfectly suits your taste in sound right out of the gate. Like everything else, each piece of gear has its own "sound", but most of the better designed gear that's set up to deal very well with jitter from USB (or....firewire) will sound very good, if not exceptional. Regardless of what others may say.
And....depending on the DAC or implementation of jitter control, cables may matter. Even USB cables. There....that will open more argument to this thread.