Downside to "revealing" system?

One thing that should not be overlooked is the panel/woofer interactions with the room.

I strongly encourage you to consider reaching out to GIK acoustics for help.

I relate "Reveling" systems to my experiences at symphony concerts.
Balcony center vs. way back at "standing room".  Now this is only my experience at the Neil Blaisdell Concert Hall in Honolulu. And I have never sat Balcony Center.  But 7 seats off.  Yes.  Large 3d  soundstage. Pin point placement of instruments.  I could focus and hear individual instruments.  Page turns.  The air blowing into a flute.   As I recall in school, the only better?, is at the conductor's podium.  As we all had to take a turn there a time or 12.  ANALYTICAL.  I personally don't want to be there for a concert performance.  On the other hand....

I had a unfortunate but fortunate circumstance of Standing Room Only.  Small image.  No pin point resolution.  But What a Wonderful Sound!  OMG  Acoustics and blending of notes I had not heard seated anywhere else.  I told this to a friend who performs with the orchestra and fellow audiophile.  He often would  go up back there and listens.  Concurs.  "You wouldn't think so,  huh?"

It feel it is what your want or goal for your reproduction system.
Compromises and Trade Offs.

Majority of the time I just want to enjoy the music.  Home or Hall.
  

I feel a cartridge or speaker(s) in particular can often create a too analytical presentation (not to be confused with highly detailed and resolving) but including a cart that focuses on record surface noise, pops and clicks is definitely not for me. With the latter it’s often stylus type or merely a set-up shortfall that is the problem.

In my experience it is all about the speaker (character) and that’s often where the issue lies. Keeping in mind that a speaker/room/power amplifier(s) matching is absolutely critical. 
In the finer tuning of a kit sounding true to music timbre and being tuneful, having enjoyable colour and pace, realistic sound space, should all be present and in good sonic balanced reproduction. 
The problem remains, any mismatch in components, any room/speaker interaction, any set-up missteps could collectively drive the performance to a focus on “the analytical” or an exaggeration of portions of the performance, out of balance regardless of each individual components competency, reputation, or expense. 
Tweaks are tweaks (like equipment placement, burn-in/settling, or cleaning up the mains), and though collectively they become a very, very big deal refinement wise, they are not in the fundamentals (laws of physics being one) and can’t possibly fix something that does not have a solid Gestalt impo.

Finding an enjoyable solution is our quest but often comes hard, from personal experimentation based in science, from a trusted knowledgeable friend or three with ears, or a good audio dealer if you can find such a thing. Audio blogs have extreme limits in helpfulness and can lead one down numerous rabbit holes. The great propensity is to just keep spend more money and buy different stuff. 
To me, much of this hobby/pastime just sounds a lot like electronic gadgets making noise, not so much like sitting down to a good musical performance. That’s cool unless you have a limited amount of time, or money, or just really love spending your time listening to music.

The difficulty of making bad vinyl recordings acceptable is that the compromise may adversely affect good vinyl recordings.  Whether the compromise is acceptable is highly subjective. 

Maybe you can live with a compromise cartridge.  Or maybe have a separate forgiving cartridge for the poor vinyl recordings.  A Multiple tonearm turntable may also be a good idea in your situation.  

Swapping out other components (phono stage, preamp, amp, speakers) can be very expensive.  Seems that changing your phono stage may be most economical fix (besides exiting this hobby).

I seem to agree with much that has been expressed above toward the idea that beyond subjective tastes, and not to go down the elitist rabbit hole of spending more and more, to (possibly) get incremental improvements. There are two things that ring true from my experience and what has been repeatedly expressed. 
...for it’s easy to be good, and it is much more difficult to be great. 

1. That with a properly set-up turntable, we should only get very good results across the board from any recording. Granted, I believe that a highly resolving kit will start to reveal short comings in every aspect of the recording and playback process. There is no way out of this. And if that disturbs you, then reality will be an unrelenting and brutal mistress. You would doomed to eternal unhappiness and frustration, OR poses a system and software that is dumbed down and removes us viscerally from the joy of music. Why bother?
That said, for me a line-contact stylus with special attention to aligning SRA seem to repeatedly have superior benefits in playing all kinds of vinyl. 
2. This can be a somewhat difficult but very rewarding hobby if we have the tools to pursue it. We need resources of experience, knowledge, patience, and a commitment to looking toward this as a lifetime investment in well reproduced music for all the joy that it will reward us with. 
Jim Smith’s book Get Good Sound and his subsequent videos and interviews offer some real practical and experience driven knowledge of good speaker/room set up.
I believe we would be able to help you better if we had more information from you. With all due respect, this is like telling a doctor that it hurts, please fix it... There are lots of good people on the site with plenty of good advice honed from decades of diverse experience. I would advise tapping on it. 
No offense meant joeyfed55. It sounds like you could find enormous utility in having someone with lots of experience confirm your analog set-up, they can be tricky, and confirm your room/speaker relationship just mho.