Why have capacitors improved so much over the years?

Redwood, it’s just a matter of getting used to them. Capacitance is an effect of two conductors separated by an insulator. The bigger the plates (conductors), the more capacitance. The closer together, the bigger the capacitance.

Accordingly, the capacitors are made so that every other plate is connected together, and the shaft turns the plates so that they become close together and alternating. This increases capacitance to the maximum value, while turning past that place decreases the capacitance towards the minimum.

With the help of a meter you’ll figure it out in a few minutes. Nothing exotic here, the theory’s been known for 200 years.

Since you seem to be interested, one of the best books you can buy is an edition of Art of Electronics by Horowitz and Hill. First edition 1980, third edition just a few years ago.

I have no idea what I’m even looking at with those air caps!

No one does. Nor does it matter. What matters is how they perform. As determined by listening. Which is what I said in the beginning.

If the real question is not why but which this is how you go about finding your caps- study this. Everything you need to know. http://www.humblehomemadehifi.com/Cap.html This was my main resource for upgrading my crossovers. (Thank you, Rick.)

While the referenced page is extensive, it's also a lot of hooey as components perform in concert with their containing circuit. There is no absolute and cost is seldom an arbiter of quality.

More than once, "Wow!" devolved