As Michael Broughton observes, the American city is named after the English town, which is pronounced the same way.
The first part isn't pronounced like "Ply" because it's not the same word and doesn't have the same historical origin. The second part does appear to have originally been the word "mouth", but because it is in an unstressed syllable, it appears the vowel sound has evolved differently in the place-names ending in "-mouth" and the independent word "mouth". James posted a link to a very relevant Wikipedia article, the Toponymy section of "History of Plymouth". According to Wikipedia:
The modern name has two parts: Plym and mouth. The element Plym is taken from the River Plym along which it traded with its parent settlement of Plympton, but its name (first recorded as Plymentun in c. 900) is considered to derive from the Old English word for 'plum tree', also ploumenn in Cornish, though the local civic association suggests an alternative derivation from the Celtic Pen-lyn-don ("fort at the head of a creek"). An alternative derivation is from Latin plumbum album 'British/white lead' -meaning tin - where plomm is also Cornish for lead.
By the early 13th century, the river was being called the Plym
(Plyme, in 1238), as a back-formation from Plympton and Plymstock
(first recorded as Plemestocha in 1086). The earliest records of the
name Plymouth date from around this time (as Plymmue in 1230, Plimmuth
It looks like in spelling, the double "mm" eventually was reduced to a single one, but this was not accompanied by a change in pronunciation.
The single "u" was also later replaced by "ou"; this may have occurred at a time when the vowel was pronounced like the vowel in the independent word "mouth," or it might have just occurred by analogy with the spelling. The reason the vowel is not currently pronounced the same as the vowel in "mouth" is likely due to the process of vowel reduction that applies to many unstressed syllables in English.
A fact that supports the idea that this is related to stress was brought up in the following comment by JHCL:
In Britain, there are a few such place names that rhyme with "South"
as commonly spoken (for example Avonmouth, Lossiemouth, Cockermouth),
but as far as I can tell they all seem to be where the "-mouth" suffix
comes after an unstressed syllable. Most, as with Plymouth, Dartmouth
and Bournemouth, are said with a schwa. (Although I've heard
Tweedmouth pronounced both ways.)
Syllables in English words have a tendency to alternate between being stressed and unstressed, so it makes sense that reduction might occur in syllables directly following stressed syllables, but not in syllables directly following unstressed syllables.
In general, the spelling and pronunciation of English placenames are often fairly remote from each other (even more so than for ordinary words). Cobaltduck linked to a Wikipedia article with a list of unintuitively pronounced place names that are even "worse" in this regard.