Will increasing size of amp improve sound quality?

matching the amp and the speakers is the most important. i would think your rotel set is fine.

I would look to improving your speakers first.

Do you have a suggestion for upgrading the speakers?

With your current speakers -- no.
As to upgrading speakers take a deep breath until you can go for Sonus Fiber Grand Piano.
With your current amp they will sound yummy.

Its true only if you have big woofers to get them moving or very insentive or low impedance speaker. I agree that the book shelves don't require any more power, get some bigger Klipsch. If you are a Klipsch fan you can pick up vintage Heresys with a 12 inch woofer (only the original heresy type 1 not the 2= pre 1982 for about $400 in medium condition) they like a little juice. Yes I know they are 94db per Watt at 4 feet (not meter) but its all midrange horn with juice, the woofers really give you their capability, great but somewhat rooled off and very tight bass its enough for me. Not good for for organ Music or classical if that's your preference. They will rock your heart out if you ask them to.