How many of us have actually started assembling a system from absolute zero--without a single component carried over, gifted, inherited, or whatever? In that rare case I'm going to say speakers, because (a) room size, speaker size and appearance, are much more variable and need more accommodation than electronic component size (17" anyone), and (b) all other things being equal*, variation in speakers makes a more immediate impact on overall size. (In a reasonably balanced system, where $$ are equitably spread over the different components.)