There was a single word for that but I couldn't remember what it was. I'm using it in a piece of fiction as a way to describe a character. I'd appreciate the help.
As already suggested by the learned members 'remnant' and 'Hellion' in the earlier answer and comments, both 'savant' and 'idiot savant' could be ideal expressions for your purposes.
It seems that 'idiot savant' (literally meaning 'very knowledgeable idiot' from the French) would be a paradoxical intensification of 'savant', because the sense of being less competent in most areas other than that of special ability is already embedded in the meaning of 'savant' as quoted in the earlier answer by remnant.
However, there seems to be rather a fine difference, in that a 'savant' would be very learned at something but not necessarily idiotic in all other matters; whereas an 'idiot savant' is indeed an extreme form (of savant) in being exceptionally weak at everything except the special talent! Some people consider that 'savant' carries positive connotations as it simply means 'highly gifted/ learned in some specific area', but 'idiot savant' carries negative connotations because of the assumption of extreme ignorance/ incompetence in most areas except the gifted field.
On the other hand, 'specialist', while certainly implying special expertise in some area, does not create the sense of bring incompetent in other fields.
Definition of savant (by Merriam-Webster online dictionary)
1: a person of learning; especially : one with detailed knowledge in some specialized field (as of science or literature)
2: a person affected with a mental disability (such as autism or mental retardation) who exhibits exceptional skill or brilliance in some limited field (such as mathematics or music); especially : autistic savant
Merriam-Webster defines 'idiot savant' as follows (originally quoted by the senior member Hellion in comments)
Definition of idiot savant
1: savant (see above definition)
2: a person who is highly knowledgeable about one subject but knows little about anything else (which corresponds almost exactly to your own description of the word you seek!)
"Matt is just your everyman most times, and quite competent at most things; but ask about the etymology of obscure English verbs on ELU and he becomes a savant!"
Joe is unbelievably good at writing brilliant software for any situation, but disastrous at the simplest task otherwise: he is (almost) an idiot savant.
As a single word, you might consider specialist.
one who specializes in a particular occupation, practice, or branch of learning
Or, savant. From Cambridge Dictionary:
a person with a high level of knowledge or skill, especially someone who is less able in other ways
There is also one-trick pony, which closely matches your requested meaning, but is more informal and not a single word.
Definition from Merriam-Webster
one that is skilled in only one area