Gh is a digraph in English (and in some other languages). In English, you can see it at the beginning, in the middle and at the end of the word.
If ⟨gh⟩ is not at the beginning of the word, it is almost always either silent or pronounced as /f/. (silent if in the middle --> light, /f/ if at the end --> tough)
Some exceptions are:
lough (and certain other Hiberno-English words) where ⟨gh⟩ is silent. [⟨gh⟩ historically represented [x] (the voiceless velar fricative) and it still does in some words.]
Edinburgh - ⟨gh⟩ is occasionally pronounced [ə]
Note: The locals pronounce it like "Edinburrah" and all the other "burghs" (Fraserburgh, Musselburgh etc.) rhyme. But there is an exception to this exception which is Pittsburgh (well, it is in US and not Scotland but shares the same suffix "burgh"). Another good question that comes to mind is "Why doesn't Pittsburgh rhyme with Edinburgh?" and there is a good answer in Quora if you want to check.
When gh occurs at the beginning of a word in English, it is pronounced /ɡ/ as in "ghost" and it does not derive from a former /x/. One might ask where it derives from. So, I checked the etymology of ghost and found this explanation in Etymonline:
The gh- spelling appeared early 15c. in Caxton, influenced by Flemish and Middle Dutch gheest, but was rare in English before mid-16c.
After all the information I provided, the main question is:
- Is there any English word starting with "gh" and "gh" (at the beginning) is not pronounced as /ɡ/ ?
I couldn't find any exceptions but English has a lot of surprises. There might even be a dialectal exception.
For example, I thought ph is always pronounced as /f/ when it is at the beginning of the word. But there are some exceptions like phthisis, phthisic, phthalate which I saw in this El&U question.