Which way do I lean...

not so with horns, at least the Heresy...

At this time I'm not trying to buy more speakers - although we will see how that works out for me.  I have Watkins Generation 4 on each side of my desk.  They sound best maybe 8-20 feet away.    I was not thinking of re-purposing them, although they are great about anywhere, I could use a smaller speaker on the desk like ProAC Tablettes (or look at ones you and others suggest), but will stick with the Watkins for now, most likely where they are.

It was really the Heresy I have some bookshelf space in another room for if they are sideways.  In that same space I could use the much smaller ML 35XT, or the 35XT sideways ear height on another shelf.  That may be my final config.  

Simply because the Heresy has a stand incorporated on bottom does not mean it can’t be turned to its side. C’mon, people, where’s your creativity?! ;)

I find it so refreshing, so interesting to turn speakers on their sides to create a Landscape orientation versus the typical Portrait orientation, that I have even done so with 4’ tall speakers such as the Daedalus Ulysses, Magneapn .7 and PureAudioProject Trio15 Horn1 (my favorite in Landscape orientation)!

It costs nothing, harms nothing, and is informative and fun! Go for it! While at it, try upside down too! Some speakers sound better inverted, even though the designer made them conversely. :)

Oh, yes, and play with the baffle slope, the angle of the front, to suit your taste. 

Is it "OK" to put speakers on their side?

Yes of course you can do it, but think of this the prototype speaker was voiced with hours/days of xover/stuffing even driver and maybe box size changes by the manufacturer to sound that way standing up, and then put into production.
Your going to be throwing away some of his hours spent voicing it, because it will sound different to what he wanted you to hear.

Now that I've made you feel real lousy for even thinking it, don't worry about him just do it.🤔

Cheers George