Newbie to the awesome audiophile world

It's going to take time and effort to sort through the possibilities and one of the difficult parts is sorting out the good information from the bad on forums like this. Go slowly, read as much as possible of the often conflicting information and advice you'll come across and trust your ears. Resign yourself to not getting it just right the first time, which argues with sticking with mainstream brands with excellent reviews that have good potential resale value, possibly used. 

Just way too many possibilities and variables to even begin to put together a useful answer.
Instead I suggest you read a lot of the current threads and get a feel for what others are doing/ have done.
Good luck and stay safe.

You say you want high fidelity and not just impressive sound. One idea that I have always gone back to: source, source, source! Always remember, any information lost or distorted at the source cannot be retrieved or corrected further upstream. Oversimplification to be sure, but the idea has been essential to get closer to my idea of what high fidelity is. Good luck and have fun.

Very FIRST thing you should do is go LISTEN TO LIVE unamplified music!Doesn’t matter where or what,coffee house,public street,wherever..Go hang out at Music Center & listen to the different instruments being played around with.Learn what music sounds like coming from real instruments so you have a good foundation to build a system on..
I suggest you start with a good,all in 1 solution that includes a DAC with USB & Toslink inputs,maybe even a phono stage for future turntable use(though I don’t recommend vinyl)..This unit will also include a built in power amplifier so all you need do is add speakers,plug a few cables in & away you go..
Have a look at the Quad "Vena II",Creek "Evolution 50a with Ruby DAC board",maybe a gently used Hegel..
Speakers are difficult,you need to listen to many different types to learn what sounds good to you..It has been my experience that you can NOT go wrong with speakers from Dynaudio or Sonus Faber but I also like Dali & if I were you that is where I would go for speakers..Dali use wood pulp fibre for midbass driver giving a wonderfully lucid midrange hard to find at entry level prices along with very smooth but not rolled off silk tweeters..If your going to go with a metal tweeter make sure to hear them with your amp of choice in your room if possible.I just don't like metal anything in speaker drivers..

Thanks everyone for providing such great inputs. I’ll take it slow and try to learn as much as I can before buying.

A little more about my music preference: I listen to a wide range of music from classical piano, jazz, latin jazz, rock, pop and trance. So I would need speakers that can handle a broad spectrum and provide neutral accurate sound. For starting out i'm thinking pairing a nice pair of speaker with a good integrated sound. My source music is coming from youtube and I do have a library of high quality wave music as well (50 MB per song). 

I'm currently looking to wet the appetite with more of an audiophile (used speakers). Revel, Monitor Audio, Bowers & Wilkems, KEF, paradigm are a few brands i'm interested in. I'm thinking getting an stereo integrated amp to pair with the speakers for now to have something to listen to right away and build up later on. I've heard lots of good reviews on Peachtree Audio nova 300 or Rotel, classe lines of integrated amps.