If you have a problem with using a dust cover and it is not just in your head it is because your turntable and/or dust cover are designed incorrectly. In either case you get inferior sound particularly if your turntable is in the room with your system.
The Dust cover can not be mounted to the platform carrying the platter and/or tonearm. The SOTA Cosmos is an excellent example of a turntable beautifully designed for a dust cover. The dust cover is mounted to the external chassis. The sub chassis is suspended inside with the platter and tonearm mounted to it. The SOTA sounds better with it's dustcover down because all the other sound in the room is attenuated over 20 db. The platter, tonearm and cartridge are better isolated from the environment. With the SME there is no external enclosure to mount a dust cover to so I sit the turntable on a platform to which the dust cover is mounted. The dust cover does not touch the SME.
If you have an unsuspended plinth to which the platter, tone arm and dust cover are mounted the dust cover may transfer some vibration at resonance frequencies to the active parts of the turntable making things sound worse. These tables and designs like the VPIs and Clearaudios need to be set on a platform to which the dust cover is mounted then you will improve the sound with a dust cover and prolong the life of your records, tonearm and cartridge.
This is obvious and the case unless you hear micro vibrations like lewm.
In which case your system is always going to sound lousy and you might as well give up and play cricket.