What prevents DIY speakers to become the ultimate?

Northcreek notwithstanding, a non-dedicated non-OEM can't buy tight-production tolerance drivers required for superb-imaging, ultranatural response. As alluded above, the best manufactures have custom-made drivers specced within a tight sensitivity window, and even then sometimes pair-match them. (Verity even jeeps a response curve on each midrange for replacement if needed!)
It's a given that Madisound, Solen, etc, get the outer 2/3 or so of a given driver run! I've built a couple of 2 and 3-ways, and once I heard a midrange voicing change of only 1/3dB over 1.5 octaves during a crossover massage, I gave up, knowing that I'd NEVER be able to afford drivers that well matched! This is by far the largest impediment to DIY
ULTRA-high quality production. Though a reseller can sit there and wade through a box of drivers, sorting their sensitivity with a Radio Shack SPL meter, and then selling so-called "matched pairs", that's only going a short way to providing yje standardization required, 'cause you gotta do it with the crossovers next!
Tweeters and caps are notoriously wayward in spectral response. When I heard Verity Audio's midrange, I cried, and now have a couple crates of nice big inductors, caps, resistors, etc., if anyone wants 'em for a dime on the dollar! Spendor, Revel, Snell, and even Boston (!) do a VERY creditable job of pair-matching and building to a target reference with decent precision. Who's going to build us DIY'er some clones to work with? I'll stick to power cords and other non-transducer components....
My hatsoff to those still trying!

Although tempting,I passed on DIY kits...it is hard to justify spending $500-600 on a pair of DIY monitors when you can buy a very good pair of used monitors for that kind of cash...such as the SPendor 3/5,QUad 11L,etc...and be certain of the outcome.. today even $500-600 new will still get you very good sound... w/ alot less headaches...

I spent about $1400 on a pair of DIY speakers, and am extremely pleased with the outcome. I built a set of Voigt Pipe cabinets from oak, and loaded in a set of Lowther EX3 single drivers. The sound is exactly what I was looking for, and the job was quite easy. Didn't have to worry about building a crossover, because I didn't need or want a crossover in my speaker system. Just one, fullrange driver per channel, with point source radiation. No problems, very happy, saved money.

In my experience (twentysomething years of amateur speaker building), crossover design is what primarily separates the men from the boys. Other significant factors can include custom-tailored drivers and exotic cabinet materials.

Note that in my twentysomething amateur years I've amassed perhaps as much experience as a professional does in his first year or so. And I don't have the engineering background that a professional does.

The main practical advantage of DIY is that there are no layers of price markup, and the labor is free (or nearly so). This means that DIY speakers can include a lot of painstaking detail work (especially on the cabinet) that would be prohibitively expensive for a manufacturer.

For me, the appeal of DIY is the joy of creating something that no one else has ever created before. I'm tweaking a high efficiency monster in my living room right now. Yup, it's R&D for a (hopefully) commercial product, but it's also a lot of fun. Assuming my prototypes validate the concepts I'm exploring, I'll engage a professional designer before going commercial.

So, can a DIY design be the ultimate? In 1998, a first-generation commercial effort by longtime amateur designer Siegfried Linkwitz walked away with Stereophile's "Speaker of the Year" award. The Audio Artistry Beethoven was essentially a commercial incarnation of a DIY design, and in fact you can go to Siegfried's website and learn how to build your own Beethoven equivalant, called the Phoenix. I would suggest that amateur speaker builders in the same league as Siegfried Linkwitz have a shot at designing and building a truly "ultimate" loudspeaker. The rest of us are somewhere farther back on the learning curve.