Many say that "prefer X to Y" has a more formal ring to it than "prefer X over Y". Are there any dialects where you wouldn't use "prefer X to Y" in colloquial speech at all? Conversely, are there any manuals of style that discourage using "prefer X over Y" in formal writing?
The Merriam-Webster Dictionary of English Usage (MDEU) suggests that to is the ordinary word used to construct comparisons using prefer: “when it is used to compare two things in the same sentence, the second […] is usually introduced by to.” They note that over (“Nine out of ten dentists prefer Crest over the competitors”), and rather than (“He prefers to stand rather than to sit”) are also used. Rather than is especially helpful when the compared items are infinitive clauses beginning with to, therefore avoiding the problem of too many tos ( * “He prefers to stand to to sit”). Above can also be used (“Prefers this brand above all others”), although they note that above, along with before, were noted by the OED as being used formerly.
MDEU notes that some commenters have criticized constructions with than and rather than, and suggests ultimately that plain than (“He would have preferred to fast than carry it”) is awkward because it is unfamiliar.
With respect to colloquial speech, I think that in some informal registers you wouldn’t use the verb prefer at all (which is a formal word), and would use like and a compatible syntactic comparative construction: “I like sleeping more than working”
A: Would you like a ride?
B: No thanks, I prefer to walk.
You can't use prefer over in this case. Over is used when there are two clear choices in the phrase.
Think of over as setting a list of preferences and putting one over top the other.
I prefer jogging over running and walking.
I prefer fish over beef and chicken.
In these two examples, the meaning could be slightly ambiguous and sound like you're comparing two things, not three.
I prefer jogging to running and walking.
I prefer fish to beef and chicken.
My brain does not differentiate the two:
"I prefer sleeping to working"
"I prefer sleeping over working"
Sounds the same to me- perhaps "prefer/to" flows slightly better. On the other hand "prefer/to" is more likely to be ambiguous:
"I prefer sleep to work"
In conversation the word "rather" seems more common than "prefer" but others may have different experience. Either way- I would be shocked if anyone criticized your usage of "prefer/to" versus "prefer/over" in any unambiguous context- formal or not.