There are five sounds that we classify as /t/ in English: the t's in "top," "stop," "pot," "potter," and "button." They aren't identical, but we do think of them all as /t/. You can test some of the difference by putting your hand in front of your mouth as you say them; the amount of breath coming out is different. These different sounds all classified the same are called "allophones." Google "english allophones" and you can find more on this.
One useful resource is academia.edu's English Allophonic Booklet. Note that /p/ and /k/ do actually have allophones as well. Some of them may be hard to distinguish, but I can hear the difference in the p's in "pal" and "lap."
Why don't all phonemes have the same number of allophones? For one thing, the sounds occur in different parts of the mouth or in different ways. But also: language is arbitrary. Add "s" to an English word and it will be pronounced /s/ ("cats") or /z/ ("dogs"). Add it to a Spanish word and it's /s/ either way.