B&W N800, 800D, 802D for my room 16x17x8

I've had both N and D in and the N is somewhat harder to drive on the low end. As far as is it worth it to pay the extra, that is a matter of opinion and could you use that extra to better something else. If you're willing to buy new they have a new series coming out that I would have to hear before getting the old. I had the 804N then the 802D and now have switched to Wilson. I really worry that your room isn't big enough for either of the choices but I would say you have to go with 802 with that room size and spend the $$ on making sure you have 300 to 500 watts of good power to drive them, not for loudness but control and stage is better when you have the juice on these speakers.

Have you looked outside the B&W domain?

The 802D would be perfect for your size room, though I think it can also handle the 800D. I don't have much experience with the Nautilus series, but my understanding is that the Diamond tweeters were a marked improvement from the previous models.

I use 802N's in a 13 X 24 room with 300 watts, they are still a great value here in the used market. Your the room size itself could pose more of a problem because the two walls are equal length. Make sure the sound accoustics are in place first because without proper treatment any speaker will not perform to it's capabilities.