In Kipling's story 'A Sahib's War', a Sikh is caught up in the Boer War, and says
Kurban Sahib appointed me to the command (what a command for me!) of certain woolly ones--Hubshis--whose touch and shadow are pollution. They were enormous eaters; sleeping on their bellies; laughing without cause; wholly like animals. Some were called Fingoes, and some, I think, Red Kaffirs, but they were all Kaffirs --filth unspeakable.
(Elsewhere he says "Do not let him herd me with these black Kaffirs"; Hubshis are a tribe of Black Indians, presumably used here where a European would say Black.)
So evidently by the turn of the century Kaffir was used in South Africa as a term for non-believers, as a specific term ('Red Kaffirs') for tribes who were non-Muslim among Muslim neighbours (compare Kafiristan, the only Afghan province which was non-Muslim), and as a term of abuse; the descending slope is obvious.
PS Yes, he is fictional, but I have considerably more faith in Kipling's research into Indian use of language than in most historians'.