Yes, it’s not necessary to spend big bucks to enjoy great sound with vinyl. However, it can get better with more expensive gear.
My view is that vinyl can be better not necessarily because it’s analogue, but because it’s mastered better.
I have found that what’s good about vinyl still persists after the analogue signal is transferred to digital and back. In other words the AD and DA conversions can be transparent. With the Linn Urika Ii phono stage, doing some of the RIAA processing in the digital domain is beneficial, presumably because distortion and noise is reduced.