I think the definition changes as you enter and travel along the hifi path. When I first started I thought my humble, low cost system was bloody great and truly hi-end! Of course, as I travelled down the hifi path, and ultimately, the upgrade path, things became clearer.
You never have the best system. There is always a better one, and when you hear it, it is apparant. Then your perceptions alter again re mid-fi/hi-fi/etc.
Just enjoy the ride, listen to as many systems as possible, and never close your mind to the fact that some ultra piced items deliver truly astonishing performance. Even if they are beyond reach, they are well worth listening to, as they improve/hone your perspective.
To suggest that the high end is reserved for crazy wealthy people is ridiculous. By defintition this is unlikely. It is also grossly incorrect. I know a few guys who have megabuck systems, highly modified, and forgo lots of things in life to attain their hifi asperations. They are niether wealthy, nor crazy. But they are very happy!
As am I.
Just enjoy your new hobby, it can, and should, last a lifetime.