I'm trying to describe evil magic and creatures to my players and to set the tone, I'm trying to use Middle and Old English words and phrases. After googling a while I couldn't find a definitive answer if Fell and Fel are both correct spellings of the same adjective?
Both are attested
Before Modern English, there were really no overarching prescriptivist entities, so the concept of "correct" spellings didn't exist. What we have are attested spellings, sometimes numbering in the hundreds for a single word.
The Middle English Dictionary is the best free resource I know of for seeing attested Middle English spellings. For fel (adjective), it shows quotes with both fel and fell spellings (and a few more like felle, double f spellings — equivalent to capital F, and even feolle).