Almost any all-tube phonostage will have substantially more background noise than one not using tubes for the hi-gain sections. Among the all-tube phonostages, the Premier Fifteen is not reputed to be one of the quietest on the market, although it's well-regarded sonically. As far as the tubes themselves go, there is no user-biasing involved with tube products other than power amp output tubes, so you needn't worry about making adjustments.
However, the total background noise displayed by tube preamps will be greatly affected by the individual tubes being used. Although there is no way to know for certain whether at this point your tubes are "worn out" or not (preamp small-signal tubes can last a very long time before performance begins to degrade noticeably) - and anyway "worn out" in the case of these tubes does not necessarily in and of itself mean increased noise - investing in some carefully tested and screened lowest-noise-rated tubes (typically best 10% or so of those tested) from a professional, reputable, and satisfaction-guaranteed source would be warranted in your case (the noise level of the input tubes being particularly critical). These could be either new production tubes (maybe bought from C-J themselves) or "NOS" (new old stock) tubes from one of the firms specializing in that area (expect to pay more for these, and I won't go into the possible pro's and con's of buying NOS, as I'm sure other members having stronger views and experience with NOS will post after me).
Also, it is true that since your preamp was made, C-J has introduced a Mk. II version which I believe may supposedly be a little bit quieter, and I think an upgrade is available from the factory, but you should ask them about that.
Just bear in mind though, that if background noise like this is particularly bothersome to you, and especially if you like to listen at higher levels to music containing quiet passages, then you might not own your ideal phonostage with this model (or maybe any all-tube model) no matter what you do to it, especially in the case of someone running a fairly low-output cartridge such as your B-M LO.4. You could consider adding a step-up transformer (and maybe modifying the P-15 to lower its own gain) in order to increase the S/N ratio, but your cart is not so ultra-low-output that this would be a necessity with all phonostages, and that can get kind of expensive vs. just switching to a quieter phonostage.
OTOH, the gain figures you mention for both of your phonostages are on the low side for your cart to begin with - I think you could probably stand to have at least another 6dB-10dB+ of quiet gain in total, whether from a step-up or a phonostage - so if you're having to turn your preamp's volume control way up in your system to listen to records (relative to other sources), and your tubed Counterpoint preamp is not the quietest in the world itself (and/or if your speakers are fairly sensitive - I'm not sure about this with your M-L Odyssey's), that combination would also tend to create systemic S/N ratio problems concerning phono for the gear you have assembled. Matching cartridge output to the gain structure of the amplification chain becomes pretty critical with lower-output cart's, and using tubes for the hi-gain stages makes it even more important that this be done optimally in order to get the analog performance you paid for. I'm not saying it can't be done using all tubes, but it has to be done right.
However, the total background noise displayed by tube preamps will be greatly affected by the individual tubes being used. Although there is no way to know for certain whether at this point your tubes are "worn out" or not (preamp small-signal tubes can last a very long time before performance begins to degrade noticeably) - and anyway "worn out" in the case of these tubes does not necessarily in and of itself mean increased noise - investing in some carefully tested and screened lowest-noise-rated tubes (typically best 10% or so of those tested) from a professional, reputable, and satisfaction-guaranteed source would be warranted in your case (the noise level of the input tubes being particularly critical). These could be either new production tubes (maybe bought from C-J themselves) or "NOS" (new old stock) tubes from one of the firms specializing in that area (expect to pay more for these, and I won't go into the possible pro's and con's of buying NOS, as I'm sure other members having stronger views and experience with NOS will post after me).
Also, it is true that since your preamp was made, C-J has introduced a Mk. II version which I believe may supposedly be a little bit quieter, and I think an upgrade is available from the factory, but you should ask them about that.
Just bear in mind though, that if background noise like this is particularly bothersome to you, and especially if you like to listen at higher levels to music containing quiet passages, then you might not own your ideal phonostage with this model (or maybe any all-tube model) no matter what you do to it, especially in the case of someone running a fairly low-output cartridge such as your B-M LO.4. You could consider adding a step-up transformer (and maybe modifying the P-15 to lower its own gain) in order to increase the S/N ratio, but your cart is not so ultra-low-output that this would be a necessity with all phonostages, and that can get kind of expensive vs. just switching to a quieter phonostage.
OTOH, the gain figures you mention for both of your phonostages are on the low side for your cart to begin with - I think you could probably stand to have at least another 6dB-10dB+ of quiet gain in total, whether from a step-up or a phonostage - so if you're having to turn your preamp's volume control way up in your system to listen to records (relative to other sources), and your tubed Counterpoint preamp is not the quietest in the world itself (and/or if your speakers are fairly sensitive - I'm not sure about this with your M-L Odyssey's), that combination would also tend to create systemic S/N ratio problems concerning phono for the gear you have assembled. Matching cartridge output to the gain structure of the amplification chain becomes pretty critical with lower-output cart's, and using tubes for the hi-gain stages makes it even more important that this be done optimally in order to get the analog performance you paid for. I'm not saying it can't be done using all tubes, but it has to be done right.