Early etymology of boy: Wikipedia has:
Etymology:
The word "boy" comes from Middle English boi, boye ("boy, servant"),
related to other Germanic words for boy, namely East Frisian boi
("boy, young man") and West Frisian boai ("boy").
Although the exact etymology is obscure, the English and Frisian forms
probably derive from an earlier Anglo-Frisian *bō-ja ("little
brother"), a diminutive of the Germanic root *bō- ("brother, male
relation"), from Proto-Indo-European *bhā-, *bhāt- ("father,
brother").
The root is also found in Norwegian dialectal boa ("brother"), and,
through a reduplicated variant *bō-bō-, in Old Norse bófi, Dutch boef
"(criminal) knave, rogue", German Bube ("knave, rogue, boy").
Furthermore, the word may be related to Bōia, an Anglo-Saxon personal
name.
and The Online Etymology Dictionary [re-ordered]
[boy] mid-13c., boie "servant, commoner, knave" (generally young and male); c. 1300, "rascal, ruffian, knave; urchin," mid-14c. as
"male child before puberty" (possibly an extended sense from the
"urchin" one). A word of unknown origin.
Possibly from Old French embuie "one fettered," from Vulgar Latin
*imboiare, from Latin boia "leg iron, yoke, leather collar," from Greek boeiai dorai "ox hides." (Words for "boy" double as "servant,
attendant" across the Indo-European map -- compare Italian ragazzo,
French garçon, Greek pais, Middle English knave, Old Church Slavonic
otroku -- and often it is difficult to say which meaning came first.)
But it also appears to be identical with East Frisian boi "young
gentleman," and perhaps with Dutch boef "knave," from Middle Dutch
boeve, perhaps from Middle Low German buobe. This suggests a
gradational relationship to babe.
In Old English, only the proper name Boia has been recorded. ME boi
meant 'churl, servant' and (rarely) 'devil.' In texts, the meaning
'male child' does not antedate 1400. ModE boy looks like a semantic
blend of an onomatopoeic word for an evil spirit (*boi) and a baby
word for 'brother' (*bo). [Liberman]
For a different conjecture: Used slightingly of young men in Middle
English, also in familiar or contemptuous use of criminal toughs or
men in the armed services. In some local uses "a man," without
reference to age (OED lists "in Cornwall, in Ireland, in the far West
of the U.S."). Meaning "male negro slave or Asian personal servant of
any age" attested from c. 1600. Extended form boyo is attested
from 1870. Emphatic exclamation oh, boy is attested by 1917.)
So, from times not too long after its earliest use in English, 'boy' seems to have carried the unmarked sense (male child / male relation / little brother), as well as the pejorative (urchin / rascal / knave), and even approbatory (young gentleman) associations.
The problems associated with using a term that can both be unmarked, mundane and highly proper, and offensive have been found especially in the United States and South Africa [the same Wikipedia article]:
Race:
Historically, in the United States and South Africa, "boy" was not
only a "neutral" term for domestics but also a disparaging term
towards men of color; the term implied a subservient status.
The article goes on to describe diachronic changes in the perceived degree of offensiveness of the word:
The use of the term "boy" to describe men of color has not always been
used as an insult, however; for example, Thomas Branch, an early
African-American Seventh-day Adventist missionary to Nyassaland
(Malawi) [1900s] referred to the native students as boys:
There is one way by which we judge many of our present boys to be quite different from some of those who were here long ago: those that
are married have their wives here with them, and build their own
houses, and all are busy making their gardens. I have told all the
boys that if they wished to stay here and learn, those that had wives
must bring them. This is having a good effect on them. They stay
longer, and are more attentive to their work and their studies.
... [2010s] Multiple politicians – including New Jersey Governor Chris Christie
and former Kentucky Congressman Geoff Davis – have been criticized
publicly for referring to a black man as "boy.
During an event promoting the 2017 boxing bout between Floyd
Mayweather Jr. and Conor McGregor, the latter told the former to
"dance for me, boy." The remarks led several boxers – including
Mayweather and Andre Ward – as well as multiple commentators to accuse
McGregor of racism.
This is obviously a complex issue. A Welshman would refer to a mate as 'boyo' in just the same spirit as others would refer to mates as 'lover', 'me old mucker', 'old boy', 'my dear old thing'... and exception could doubtless be taken to all of these. 'Pal' said in a certain way is just as nasty as others. As always, the answer to misuse is not disuse but proper use. And as always, if one does not agree on a final arbiter, deciding on which is which merely rephrases the problem.