So where should the tubes be?

Tubes should go where they sound good to you and work well with your other components.  It's really about your listening priorities.  I personally enjoy tubes in all my components and have speakers that respond well to tubes.  Some people really like the dynamics of SS components while others enjoy a mix.  There is no "right" answer and it's a question mainly of personal taste once you get past some general compatibility questions.  Don't let anyone tell you that only tubes sound like music - there is an entire market who only enjoy SS components and they are not "wrong". Such blather is ridiculously off-putting.

If you want to get into tubes you should start with a tube phono preamp. That’s were the rubber meets the road. I have tube phono, tube preamp and tube amp. I have owned a lot of solid state gear but there is something about tube gear that just sounds right to me!

You might find this article interesting!
https://medium.com/collectors-weekly/could-an-old-school-tube-amp-make-the-music-you-love-sound-bett...

Never talk a man out of a tube phono stage. Especially not when Decware and Bottlehead make such excellent and reasonably priced tube phono stages.  

Yogiboy is right about they just sound right. I was at a show one time, the same system was being used to demo different tables, platters, and phono stages. They all sounded different but they all sounded great until one, it had this uncomfortable edge to it. Wasn't sure at first but after a couple minutes I turned to Chris and said what is this solid state? Yeah not completely but yeah not all tube.   

So there you go. Use tubes wherever you want music.

Preamp or phono stage.

Where tubes make the biggest impact on the sound is the amplifier.  But, good tube amplifiers also represent the highest cost in making a switch (good output transformers are particularly expensive and important to the sound), so that might not be where one would start.  Also, the matching of a tube amplifier to the speaker is critically important, so you will need to either have some experience or some help in that area (e.g., a good dealer).