What makes a speaker too big for a given room?

Manufacturers should not only know the dimensions of your listening room, but about the furniture, carpet, drapery, exactly where you plan to put the speakers, and should also conduct a lengthy interview (perhaps in the listening room over at least a couple of days) with you regarding your personal taste, how much you care about the opinions of others, and, of course, examine your finances.

Also, I recently read an essay in some HiFi mag about high quality subwoofing. Although technically maybe there isn't much below 40hz or so that should concern you, the writer was listening to mandolin music with and without his REL sub. It sounded much better WITH the sub even though logic dictates it shouldn't. It seems that a well designed and properly placed sub seemingly "charges" the air in the room with energy of some hard to measure nature. Having done this experiment myself with my trusty old REL Q150e, I experienced the same thing. If I dial out the sub it seems to drain the natural life out of the sound, leaving me in a despondent state of woe not unlike the demeanor often displayed by subless minimonitor users shuffling around their tiny apartments in tattered tweed jackets pausing only to look out the window at a full range world.

big speaker vs small speaker

less imaging performance, slower bass, less life and poor low volume performance due to larger drivers

Larger loudspeakers do not have in general slower bass,less life or poor low volume level performance and if one did it wasn't properly designed. For larger loudspeakers actually have less weakness than smaller designs which are total design compromised. Overly small drivers, tweeter covering to much range limited SPL, higher thermo compression,no low frequency and if a attempt is made at reproducing lower frequency you reduce SPL and raise distortion even more. Sure everything has its place and many need or want smaller loudspeakers but to think that the laws of physics does not apply to smaller designs shows a misunderstanding of how loudspeakers function.

Good question.

Drivers are too far apart to sound coherent in close proximity.

Also bass can be too much sometimes and muddy up the sound even
further.

Also most speakers do imaging and soundstage better if away from walls
and this can practically be harder to accomplish with large speakers in a
small room.