I'm fairly certain that words that both alliterate and rhyme exist, but a list of specific examples would depend on how you define "rhyme" and "alliterate". I think the most widely accepted examples would be things like blue, brew and cash, clash, where the words start and end the same way, but don't sound exactly the same.
Definitions of rhyme
Stress-based rhyme (match from stressed nucleus to end of the word)
Some people in the comments and answers are using a loose definition of "rhyme" that apparently only requires the ends of the last syllables has to match. A more strict definition of rhyme requires identity from the end of a word back to the nucleus of the last stressed syllable. By this definition, pairs like "pavement" & "parchment", "institution" & "initiation", "democrat" & "diplomat", "comfy" & "cozy", and "dusty" & "dirty" do not rhyme, because they don't have matching stressed syllables. See the definitions of "perfect rhyme" from yourdictionary.com.
The American Heritage Dictionary gives two definitions of "perfect rhyme":
- Rhyme in which the final accented vowel and all succeeding consonants or syllables are identical, while the preceding consonants are different, for example, great, late; rider, beside her; dutiful, unbeautiful. Also called full rhyme, true rhyme.
- Rime riche.
Going with the first, we see that it mentions stress, and also the consonants in the onset of the stressed syllable.
Avoiding identical onsets in stressed syllables
The requirement for "perfect rhymes" to have different consonants before the stressed nucleus would rule out saying that a word rhymes with itself (a question that was raised earlier on this site), or the homophonic rhymes that Jonathan Van Matre mentions.
These are "rich rhymes" or "rime riche", which have usually not been preferred in the English poetic tradition, although they are common in poetry in some other languages such as French.
Definitions of alliteration
Old English alliteration: stressed syllables starting with the same sounds
In Old English alliterative verse, apparently, the device of alliteration primarily consisted of matching the first elements of the onsets of certain stressed syllables. Words in the native vocabulary of Germanic languages tend to have stress on the first syllable, except for certain categories of prefixed words. Apparently, there has been some dispute about the importance of stress in Middle English alliterative poetry, but there is a case for it being closely related to stress.
As far as I know, the definition of "alliteration" as a device in contemporary English poetry is not so strict, but it does seem to be the case that matched onsets in stressed syllables are more prominent than matched onsets in unstressed syllables. This is pointed out in "Alliteration, assonance and perception" from "Ling 131: Language & Style" at Lancaster University; and I found a modern resouce "Linking Letters: A Poet's Guide to Alliterative Verse" that emphasizes a stress-based definition of alliteration.
My understanding is that in Old English, consonant clusters usually did not have to match entirely in order for the words to alliterate: only the first consonant had to match. So it is possible for a word starting with bl to alliterate with a word starting with b followed immediately by a vowel. The only class of exceptions that I have heard about is mentioned by tchrist in the comments below: consonant clusters of s + a plosive, like sp and st, did not alliterate with words starting with s followed by some other consonant, or by a vowel, in Old English.
This seems to be a somewhat subtle issue, but I think my initial impressions are mostly backed up by the following passage from Alliteration and Sound Change in Early English, by Donka Minkova:
Any further identity of the stressed syllables in Old English was a structurally redundant element, explicitly frowned upon in the case of alliteration on identical vowels. There is no record of a parallel negative esthetic judgement regarding cluster onset identity, but it was clearly not a feature that was actively sought. In contrast, alliteration on identical vowels and alliteration on entire clusters became specially favored by the Middle English poets.
Alliterative rhyme candidates
Words starting with the same consonant, but different consonant clusters
Based on the above, it seems alliteration between different consonant clusters starting with the same consonant, but ending with different ones would meet the requirements of both Germanic-style alliteration and perfect rhyme.
Examples: brew/blue, break/bake, beauty/booty, quick/click/kick, cute/coot, die/dry, fee/free/flee, fight/flight/fright, gain/grain, pay/play/pray, spy/spry; sprint/splint, stain/strain, sweet/seat, shed/shred, tie/try, tweezers/teasers
Rhyming words with alliterative 2ndary-stressed first syllables
If we tighten the definition of alliteration to require entirely matching onsets, I think the strongest kind of alliteration that could occur between "perfect rhymes" in English would consist of pairs where one or both of the words have secondary stress on their first syllables.
One with primary stress, one with secondary stress on the first, alliterative syllable:
- dental, detrimental (alliterate in /d/, rhyme in /ˈ-ɛntəl/)
Both with secondary stress on the first, alliterative syllable:
- calibration, calculation, combination, capitalization (alliterate in /k/, rhyme in /ˈ-eɪʃən/)
- pigmentation, pollination (alliterate in /p/, rhyme in /ˈ-eɪʃən/)
- comprehend, condescend (alliterate in /k/, rhyme in /ˈ-ɛnd/)
Most of these are pretty long words ... honestly, I don't think these examples are very poetic-sounding. But perhaps there are better examples that I haven't thought of yet.
Rhyming words with unstressed 1st syllables that "alliterate"
If we consider unstressed initial syllables as valid candidates for alliteration, we can find pairs that are a bit shorter than this.
Disyllables stressed on the second syllable that rhyme with each other:
- correct, collect (alliterate in /k/, perfect rhyme in /ˈ-ɛkt/)
- decayed, dismayed, degrade (alliterate in /d/, perfect rhyme in /ˈ-eɪd/)
- repent, relent, resent (alliterate in /r/, perfect rhyme in /ˈ-ɛnt/)
Monosyllables rhyming with disyllables stressed on the second syllable:
- bride, beside (alliterate in /b/, perfect rhyme in /ˈ-aɪd/)
- peace, police (alliterate in /p/, perfect rhyme in /ˈ-iːs/)
- died, divide, deride (alliterate in /d/, perfect rhyme in /ˈ-aɪd/)
- bold, behold (alliterate in /b/, perfect rhyme in /ˈ-oʊld/)