Those magical days when the HVAC is off

The lowest ambient noise levels I can achieve in listening room is low 30's on quiet nights, this living on relatively quiet street, traffic is the worst for ambient noise levels. Daytime 50's and above. I can only imagine what I'd hear with something like low 20's. I'd suggest using resources to improve one's home and listening area to achieve lower ambient noise levels is equal to or better than using those resources to purchase a new higher resolving component.

I fully understand and appreciate the HVAC sound issue you have raised. Here are some tips I offer, after having 11 new home constructions in Southern California, Northern Virginia, and the South Carolina Low Country. (1) During construction, specify your preferred location for the air handler, intake and output vents, and thermostat. (2) During construction, insist on knowing the wall, floor, and ceiling insulation grade. (3) For a second floor, insist on a sealed wood floor with “glue and screw” versus a nail gun. (4) Investigate carpet padding, material and method of installation before choosing the flooring. (5) In an existing room, ensure that the HVAC access points to the attic or side walls have an insulation blanket affixed to the door/crawl space. (6) Select HVAC filters that balance your need for purified air and the noise generated by air flowing through the intake vent. (7) Have your HVAC system professionally checked for air output flow and balance.  (8) Ensure your windows are well insulated to minimize outside noise and your neighbor’s HVAC systems. (9) Have your roof treated with an environmentally safe sticky substance to deter pigeons and ravens from cooing, squawking, and bouncing off windows and metal exhaust vents. (10) If all else fails, put on your best headphones and enjoy the music.   

I see some of Paul McGowen listening rooms use Mini-split AC units. They are extremely quiet. 

I don't have AC.  Love it.

Increase the # of grilles in your room if possible.  If you have 300 cfm coming out of one grille it is quite a lot louder than 2 grilles at 150 cfm.

Oversize Return air.

Keep dampers adjusted at plenum, not at grilles.