OP,
Whether it matters or not is like asking if a new (larger) intake valve would improve the power output of a car engine.
Higher quality internal components in general are going to lead to better objective performance. There is a reason why mega-buck components from the biggest names in audio refrain from using cheap power supplies and rudamentary caps etc. from China. An amp that costs only a hundred bucks and claims to be a "power amp" probably isn't; and comparing it to one that costs many thousands of dollars is going to lead to finding many differences between the two.
In the end, almost everything audio matters. However, above all your perception and enjoyment are far more important.
Whether it matters or not is like asking if a new (larger) intake valve would improve the power output of a car engine.
Higher quality internal components in general are going to lead to better objective performance. There is a reason why mega-buck components from the biggest names in audio refrain from using cheap power supplies and rudamentary caps etc. from China. An amp that costs only a hundred bucks and claims to be a "power amp" probably isn't; and comparing it to one that costs many thousands of dollars is going to lead to finding many differences between the two.
In the end, almost everything audio matters. However, above all your perception and enjoyment are far more important.