There is a difference between the pronunciation of the "v" in have and the "f" in of.
The two sentences
I have two fine dogs,
I have to fine dogs,
mean different things. (I possess two excellent dogs, and I am required to impose monetary penalties on dogs.) In all standard varieties of English, they are distinguished in pronunciation by a /v/ in the first and an /f/ in the second, and the resulting change in the length of the vowel /æ/. The change from /v/ to /f/ does not depend on the consonant following it, but on the meaning of have.
On the other hand, the pronunciation of the /v/ in "of" may change, but it doesn't depend on the meaning. English speakers have a tendency to devoice or partially devoice /v/, /z/, /ð/, /ʒ/ at the end of words. See this presentation. However, we can still usually tell that they're supposed to be voiced consonants, partly by the length of the preceding vowel and partly by context.
I can imagine whether the pronunciation of "f" in of is more like an [f] or a [v] depends on the consonant after the of, the speaker's dialect, how carefully they are enunciating, whether they are speaking formally or informally, and how much stress they put on the word of. But the pronunciation of /f/ or /v/ in of does not change the meaning of the sentence. So in of, one can think of an [f] pronunciation as being an allophone of /v/, while in have to, the actual underlying phoneme is /f/.