First, check to see if there are any audio clubs in your area. You already know the competition. Get to know some local audiophiles and see what they have interest in. Will you offer repair and set-up of systems and equipment? We recently placed our equipment in a well established audio business. The place the speakers up against the wall and I was a little taken back by that. Our components sounded fantastic but I asked why so close to the wall. The response was that the wife usually does not want speakers half way out into the living room. Our gear prices match the speakers and the streamer so that was a selling point.
You probably won't get rich in this business. Plus a lot of well established manufacturers are possibly not on stable ground either. If you are just looking to sell components, then I'd say no. If you are going to offer real full services then you could be OK.