There are several different words spelled bye — and, for that matter, also several different words all spelled by. I will therefore for the moment assume that you are referring to the most frequently encountered bye, not the others.
That’s the interjection bye, which is simply a clipping of the longer goodbye. The OED provides earlier spellings of the word bye as:
Forms: 16 b'w'y', 16–17 b'y, 17 b'wye, 17 by'e, 17– bye, 19– 'bye.
(Those numbers represent the first two digits of a four-digit year.)
There you can see that the earlier writers were conscious of the word’s origin in the longer phrase be with ye. It was clipped from God be with ye, which is a frozen phrase that, being written in the “third-person imperative” style of ancient benedictions and maledictions, uses the modally marked form sometimes called the present subjunctive or bare infinitive. We would today phrase something like that as “May God be with you.”
The longer form, goodbye, has been historically spelled in nearly as many ways as there were writers. :)
α. 15 god be wy you, 15–16 godbwy, 15–16 godbwye, 16 godb'w', 16 godb'w'y, 16 godb'wy, 16 godb'w'y', 16 godb'w'you, 16 goodb'wy, 16–17 godb'w'e, 16–17 godb'w'ye, 17 goodbwi't'ye, 17 goodb'w'y', 17 goodb'w'ye.
β. 15 god boye ye, 15 god boye yee, 15–16 godboye, 15–16 godbuoy, 15–16 godbuoye, 15–16 godbuy', 15–16 godbu'y, 15–16 godbuy, 15–16 godbu'ye, 16 godbo'y, 16 god b'oy you, 16 godb'uy, 16 god, buy, 16 godbuy'ye, 16 god buy ye, 16 god bu'y you, 16 god buy you, 16 godb'y, 16 god'b'y, 16 godb'ye, 16 god b'y you, 16 good, buy, 16 goodbuy'ye, 16–17 goodbuy, 16– goodby, 17– goodbye, 18 gudebye (Sc.), 19– guidbye (Sc.).
Origin: Probably formed within English, by compounding.
Etymons: English God be with ye, good adj.
Etymology: Probably shortened partly < God be with you, and partly < God be with ye (see god n. and int. Phrases 1c(a)(ii), and the note below), with substitution of good adj. for god n. as the first element, probably by association with other greeting formulae, e.g. good day n., good day int., goodnight n., goodnight int.
God be with you is attested as a parting valediction from at least the late 15th cent., God be with ye from at least the second half of the 16th cent.
The written record therefore shows that even though a great many spellings of (good)bye have been seen historically, at no point was there any confusion with the completely different word by.
Important Teaching Point
The take-home lesson from this that may not be obvious to learners is that English has seldom if ever deliberately chosen different ways to write two distinct words that just happen to be homophones (pronounced the same way).
The spellings and meanings of English words each have their own long and notably distinct histories, most of them having gone their own separate ways at least five hundred years ago.