When the phoneme /p/ occurs at the start of a stressed syllable in English, it is aspirated with a little puff of air, making it a [pʰ]. Contrast hospital with a [p] with husband with a [b]. Hear the difference now? It is subtle, but you can have a loss of aspiration without having to gain voicing at the same time. But people mistake this in hearing.
Per the Wikipedia article on English Phonology:
In most dialects, the fortis stops and affricate /p, t, tʃ, k/ have many different allophones, and are distinguished from the lenis stops and affricate /b, d, dʒ, g/ by several phonetic features. They may be aspirated [pʰ], voiceless unaspirated or tenuis [p⁼], preglottalized [ʔp or [ˀp], or unreleased [p̚].
- Fortis stops /p, t, k/ are aspirated [pʰ, tʰ, kʰ] when they occur at the beginning of a word, as in tomato, trip, or at the beginning of a stressed syllable in the middle of a word, as in potato. They are unaspirated [p, t, k] after /s/, as in stan, span, scan, and at the ends of syllables, as in mat, map, mac.
Because the p in hospital occurs after an s, it cannot be aspirated. Since a /b/ is never aspirated in English, you are misperceiving the loss of aspiration as a signal that you have gone from [p] to [b], but that is not what is happening. Unaspirated [p] is still unvoiced, whereas [b] is voiced.
As Professor Lawler observes in comments, it can be hard to hear the difference in these contexts. Consider the common error of spelling disbursement as *dispursement. There is not much to go on there.
So your p in hospital is still a valid allophone of /p/: [p] not [b].