Gotsouldr -
What's the power source on the USB converter? If its powered by the USB bus, then that's your problem. If its an external power source, see if you can upgrade it to a linear power supply. Boulder Cable in CO does this sort of stuff and they built me one for my Trends UD-10 and it made a world of difference (sounds the same as the battery source I had, WAY better than PC power).
I am just learning why people pay so much for clean power... it's necessary if you want the best sound.
I would also try a jitter reduction unit. I use a MSB digital director. Once you hear it, you don't want to go back.
Taker easy!
Ben