Yeah, except when a speaker (in this case, center speaker) is sitting in somewhat of a hole in the 70-80hz regigion, and someone ends up getting some more "umph" by crossing their rather large center speaker over at, like 60hz or whatever, you should most always be looking at crossing over a normal sized center speaker at "small"! Why? Yes, they don't play so low. But that's not the problem entirely. Another case, and just as important of one, is that these passive speakers can't properly deal with full range bass info efficiently enough, often leading to distrotion, clipping, and squashed dynamics. Basically, not only does the speaker have to play lower than comfortable often, when set to "larger" (or anything larger than 60-80hz really), but the amps (usually receivers) can't really control the drivers well enough in most passive crossover design models either! The passive crossover is a rather innefficient design overall. And most speakers aren't so "sensitive" as well. (some larger horn speakers help greatly).
So, the vast majority of the time (I've yet to run any center speaker as large in any setup I've ever done, and I've done hundreds!), you get better efficiency, and dynamic range from crossing over to "small". The critical part is making sure your speakers and sub(s) are coupled well (and flat ideally) at the crossover frequency, and in phase! Here in lies the problem where most people run into with speaker/sub placement and setup. They know nothing of this part...