Is damping material necessary for a speaker having such a small cabinet volume?

Wait.. Is this something you bought designed to be wall mounted? if so, follow manufacturer's recommendations. 

In a speaker enclosure, damping material lessens reflections, but also increases internal volume, so it depends.  I have seen some delightful, tiny speakers use internal wadding, to good effect. 

Best,

E

GR Research sells a cabinet wall damping material named NoRez. It has a thin layer of mechanical damping material, with a 1" layer of acoustical foam, and attaches to the interior cabinet walls with self-sticking adhesive.

Generally speaking most speakers have way too much fiberglass or polyester, whatever stuffed into them, who knows why? It might be a case of monkey see, monkey do. Excellent results, in terms of bass performance, dynamics, and open transparent sound, can be obtained by removing all stuffing save for a small grapefruit size ball, depending on volume of the speakers. For small speakers I would give consideration to removing all (rpt all) stuffing. Hollow fiber wool is the best material for such applications.

A-did you glue or caulk the 2x4 studs so no air leaks?
B-is the drywall double layer and taped on the seams?
C-is the drywall glued or caulked?
D-What's on the outside wall
E-Personally I would have used double layer drywall, liquid nails and decking screws to build the wall before even trying it out.

Box construction is vital.  You may have just made a big bass drum.  Also, if it's leaking, that will wreak havoc on the sound and probably leak noise to the rest of the house.  This is why most quality in-wall speakers come built into aluminum enclosures.  The manufacturers can easily design a speaker if the cabinet is a known source.