If you start with 0 LPs and 0 vinyl rig, right off the bat you have a decent-sized expense ahead of you. However, given the first sentence of your original post, the money (within reasonable limits) will not be the most "expensive" bit about jumping into vinyl. The "expensive" bit is the time spent looking for vinyl, fiddling with your system, reading about how to make it better, and eventually the vinyl itself. If you buy the gear used, and decide a year later you do not want it, you might take a small loss but not much of one. The other stuff (the Fremer DVD, the brushes and cleaning fluids, the time spent, etc) will all be more expensive.
I find vinyl playback to be enjoyable, but am not religious about vinyl being "better than digital." I use vinyl to discover music. I can buy lots of decent records for not much money, discover what I like, throw back the rest, and then I can go learn some more. I can also listen to what I have found. To a large extent, I do not pay lots of money for a great pressing in order to get a great original pressing of something I already have on CD or SACD. If I can find the record for $5, I might buy something I already have on CD.
Personally, I would recommend that you 'experiment with vinyl' in a completely different way. If you have garage sales or thrift shops in your area, I would try to find a used working Dual, Thorens, or other table from the 1970s or 1980s. If you buy a used cart (or spring for a new one) which matches the arm, you will get a lot of the subjective qualities you seek out of a $50 yard sale special and a $300 cart which matches (ask here or over on the Vinyl Asylum at Audioasylum.com for what would match). If you really want to go cheap, try a decent receiver from the 1970s at the same yard sale and you might find an acceptable phono stage. If after buying some thrift shop records, cleaning them, setting up the table, and setting it up again, you find joy in listening to the records you bought, and joy in finding more music and playing that, then the vinyl experience is for you and you can sell your garage sale finds at a garage sale and move up to a more expensive set-up.