It doesn't seem to me there are any rules of thumb for how to pluralize a foreign word.
The New Oxford American Dictionary reports that both lemmata, and lemmas are the plural of lemma.
If I check on the Corpus of Contemporary American English, and I search for lemma*, I notice that
- The word is used mostly in academic contexts.
- In an academic context, the word lemmata is never used; in one case, the word used is lemmae.
Other examples of lemmae held to be doublets, and furnished with catchwords, but which actually occur three […].
If then I search for lemma* in the Corpus of Historical American English, I notice that the word lemmata is used for the first time in the 1900s; for the other years, before or after, the CoHA doesn't report the word lemmata.
For octopus the NOAD reports that
The standard English plural of octopus is octopuses. However, the word octopus comes from Greek, and the Greek plural form is octopodes. Modern usage of octopodes is so infrequent that many people mistakenly create the erroneous plural form octopi, formed according to rules for Latin plurals.
Looking for octop* in the CoCA, I found out that
- Words starting with octop are used with higher frequency in fiction and magazines.
- In magazines, where words starting with octop are used with an incidence of 3.14 per million, I find only three examples of text using _octopodi; in the other cases, the used words are octopus and octopuses. I also found the word octopusan, and Octopussy (the title of a James Bond movie).
If such a quality as intelligence can arise both in human beings and in the octopusan eight-armed sea animal without a bone in its body, then perhaps there is a course and […].
A search for octop* in the CoHA gives the following results: