I don't think I've heard any reasonable person make blanket statements about either pitch control or direct drive tables. However, I have heard about specific tables and specific implementation of both; as is always the case, people can make any issue quite complicated.
There are a lot of people that did not like the sound of certain specific, iconic direct drive tables, like the Technics 1200, and even the SP10. They claim that a lot of those tables have a hard or brittle sound. I've heard that quality with some direct drive tables, but, I have no way to attribute that specifically to the drive, given that there are so many other design factors involved in any table, nor did I hear this with ALL the direct drive setups I heard. I've heard speculation that this quality is the product of the servo system constantly making small speed corrections, but, again, I have no way to determine if that is the case.
I don't think anyone objects to having pitch control, the issue is how such is implemented and whether the implementation hurts the sound. For example, with AC motors (synchronous), if you have a two-phase motor where the phase is split by a capacitor, changing the frequency of the AC to adjust speed would not be theoretically ideal because the phase splitting capacitor was chosen specifically for one frequency (60hz or 50 hz). Whether the slightly asymmetrical sinewave that results from just changing frequency really matters, is the subject of debate. Some manufacturers, like Basis, provide a controller with two channels of AC supplied to the separate phases of the motor to prevent any such problems, others do not go this approach because of the obviously higher expense and trouble (the motor has to be re-wired to work this way).
I don't have any particular issue with people adjusting speed to better match the ideal. But, I do think there are a lot of people who make absolute speed accuracy a fetish, even if the minor speed inaccuracy is well within the range of inaudibility to someone with the gift of absolute pitch. If is far more important that the table delivers stable speed that does not have small, rapid, speed variations (flutter) or the slightly slower variation (wow); if the average speed is perfect, but there is a lot of wow and flutter, the table will sound like crap.