As is well known, spelling in English doesn't correspond well to pronunciation. Despite what some people may say, it isn't a very great guide to etymology either (there is a substantial etymological component to many English spellings, but the problem is there are also spellings that disregard etymology or even suggest false etymologies, and it's hard to tell which are which unless you already know the etymology of the words or memorize a bunch of rules).
The Old French origins of "o" pronounced as "u"
It seems that in Old French spelling, "o" and "ou" and "u" often were used interchangeably for the unstressed /u/ sound that developed from Vulgar Latin *o:
The Low Romance and Old French speakers were never certain how to spell unstressed o and ou sounds; they hesitated considerably. Initial unaccented ọ, free or checked, and ǫ free, presumably were pronounced ụ in Low Romance but they could be spelled u, o or ou: botellu > budel, bodel, or boudel. On the other hand a checked initial ǫ was spelled and pronounced ǫ: portare > porter.
– A History of the French Language, by Urban T. Holmes, Jr. and Alexander H. Schutz (p. 34).
Note: the quote uses an old convention for phonemic/phonetic transcription where ǫ = IPA ɔ and ọ, ụ = IPA o, u.
The French sound /u/ could be taken into Middle English as a "short" vowel /u/ or in some cases as a "long" vowel /uː/: the conditions for one or the other are not entirely clear to me.
(The OED entry for the verb "cover" mentions that "The Old French stressed form cuevre, queuvre, of the present singular gave the English variant kever, kiver, still extensively used in the dialects.)
After sound changes:
Middle English /u/ usually corresponds to modern English /ʌ/
Middle English /uː/ usually corresponds to modern English /aʊ/ before coronal consonants (like t d s z n l r) but to /uː/, /ʊ/ or maybe /ʌ/ before labial consonants (like p b f v m). (The diphthong /aʊ/ has a somewhat restricted distribution in present-day English words that have been "nativized" and it does not usually occur when a non-coronal consonant follows the vowel within the same syllable.)
In some words, the influence of the spelling or of analogy of some kind has caused pronunciations with /ɒ/ or /əʊ/ to become more frequent than ones with /ʌ/ (e.g. "conduit", which often used to be pronouced with /ʌ/ in the first syllable, and "covert").
The possible motive to avoid long runs of "minims" in words
The use of the spelling "o" in words like "cover" therefore seems to be partly due to Old French spelling conventions. There is also a theory, that seems to be fairly well supported, that people writing English at certain periods of time felt it to be preferable to avoid sequences like "uu" (the letter "v" used to not be distinguished from "u") and for this reason preferred to used the letter "o" instead of "u" to represent /ʌ/ in this context. (I asked a question about it that has some information about the etymologies of words where "o" corresponds in modern pronunciation to /ʌ/: What's the current scholarly opinion on the "minims" explanation for the spelling of "love", "tongue," etc?)
Different words have different etymologies that don't connect to French
"cove", "clover" and "over" have different etymologies leading to their different pronunciations.
For example, as you said, the word "over" comes from Old English ofer.
This word had a "short o" in the first syllable in Old English.
Some disyllabic words that had a stressed short "a", "e" or "o" in Old English, followed by a single consonant, followed by a single unstressed syllable like "er", were subject to vowel lengthening. The exact explanation of this is a bit disputed: Wikipedia refers to it as "Open syllable lengthening", whereas I remember reading some articles arguing that the real mechanism of the change was compensatory lengthening of light syllables when schwa was lost in a following syllable (e.g. the following article by Attila Starčević attributes this explanation to Donka Minkova: "Middle English Quantity Changes – Further Squibs"). Lengthening happened more consistently for words ending in schwa (which was lost) than for words ending in unstressed syllables that survived into modern English. This is part of the reason for the "silent e marks a long vowel" convention of the modern English spelling system.
"cove" is another example of Old English "short o" developing to present-day English "long o" because of "open syllable lengthening" or alternatively, compensatory lengthening for schwa-loss.
"clover" is reconstructed as having had a long vowel already in Old English. Old English ā (long back a) often corresponds to modern English /əʊ/, except for when it was shortened (as in sorry, holiday). The OED says there was actually a shortened form of "clover" once in common use:
The prevalent Middle English claver apparently represents a form clæfre with shortened vowel (compare never < nǽfre), while the current clover represents the Old English cláfre, retained in some dialect, whence it at length spread out and became the standard form.
That said, there are a couple of words like oven, shovel that for some reason seem to have developed the sound /ʌ/ despite coming from Old English "short o". Various irregular or hard-to-explain sound changes occured in Late Old English and in Middle English. Spagirl left a comment saying that in some Scottish accents, "oven" has a different vowel quality.
Whether "cover" and "over" are ultimately related seems to be a question of proto-linguistic reconstruction
As you trace the words back to different Proto-Indo-European (PIE) forms in your question, I don't know whether the similarity is coincidental or not. It seems like that would require establishing whether PIE *op- or *wer- is related to PIE *uper; I'm not familar with the reconstruction of Pre-PIE, so I can't tell you if they are.