This is an interesting excerpt from Wikipedia:
After the invention of the printing press in the 1440s, English spelling began to become fixed. This took place gradually through printing houses, whereby the master printer would choose the spellings "that most pleased his fancy". These spellings then became the "house style". Many of the earliest printing houses that printed English were staffed by Hollanders, who changed many spellings to match their Dutch orthography. Examples include the silent "h" in "ghost" (to match Dutch "gheest", which later became "geest"), "aghast", "ghastly" and "gherkin". The silent "h" in other words — such as "ghospel", "ghossip" and "ghizzard" — was later removed.
It reminded me of an article I came across years ago, which explained the disparity between the pronunciation and the spelling of the English language, something not happening in other European languages I know, at least not to this extent. In essence, it claimed it is difficult for written forms of the language to change and follow the spoken changes, especially in English. I don't remember the reasons why, but this explanation was enough for me to understand why there are practically no rules for pronunciation.
As to why these odd spellings haven't been replaced yet, I know there are suggestions for it (not only about English, mind you, a big debate has been going on in France about the simplification of French spelling), but no decisions have been made so far.