I will say that, yes, you can hear differences between DAC's.
I don't know what other DAC's you have listened to, but comparing the Schiit Bifrost to, say the Gungnir or Yggy, it is easy to hear the difference in sound quality.
If you have compared those two and haven't noticed a difference, then I would say you shouldn't spend more money on DAC's.
Bob