"How much does driver material influence you purchasing decisions?"
It has nominal influence on my purchasing decisions. Frankly, in the real world, every material represents a series of trade-offs -- this advantage versus that disadvantage. And don't forget that cost, availability, and durability are also part of the equation along with the range of performance variables.
But, that's what speaker designers do -- they set a goal and then see what choices get them closest to their desired endpoint. And then you, as a customer, pick from the hundreds of options available on the market, hopefully selecting the speaker that gets you closest, within your budget, to what you're after.