How big should center spkr be to set to "large"?

I haven't heard a center channel yet that should be set to large (and most don't deserve to be on either).
Set it to small and let the LFE come from the sub.

Set it to small. It lacks sufficient bass extension, as you have found. There are only a very few dedicated centers worthy of being set up as "Large."

I have a center that specs down into the 30hz range and it is a stretch but I do like it better at 60hz than at 80hz, very few will actually spec at full range, even fewer can honestly acheive it.

"bigger sound right there in the middle" can be best achieved by a sub that doesn't even have to be anywhere near the middle. Set up is important but if the sub is crossed at 80hz or below and even half way blended into your system, it's gonna sound like it's in the middle.

That's gotta be alot cheaper than a center speaker that has 12" woofers

Yeah, except when a speaker (in this case, center speaker) is sitting in somewhat of a hole in the 70-80hz regigion, and someone ends up getting some more "umph" by crossing their rather large center speaker over at, like 60hz or whatever, you should most always be looking at crossing over a normal sized center speaker at "small"! Why? Yes, they don't play so low. But that's not the problem entirely. Another case, and just as important of one, is that these passive speakers can't properly deal with full range bass info efficiently enough, often leading to distrotion, clipping, and squashed dynamics. Basically, not only does the speaker have to play lower than comfortable often, when set to "larger" (or anything larger than 60-80hz really), but the amps (usually receivers) can't really control the drivers well enough in most passive crossover design models either! The passive crossover is a rather innefficient design overall. And most speakers aren't so "sensitive" as well. (some larger horn speakers help greatly).
So, the vast majority of the time (I've yet to run any center speaker as large in any setup I've ever done, and I've done hundreds!), you get better efficiency, and dynamic range from crossing over to "small". The critical part is making sure your speakers and sub(s) are coupled well (and flat ideally) at the crossover frequency, and in phase! Here in lies the problem where most people run into with speaker/sub placement and setup. They know nothing of this part...