These are considered irregular plurals from Latin and Classical Greek. Rather than me copying and pasting the source, here's the wiki link that explains it.
Here's an excerpt from the link that explains it.
English has borrowed a great many words from Latin and Classical Greek. The general trend with loanwords is toward what is called Anglicization or naturalization, that is, the re-formation of the word and its inflections as normal English words.
Many nouns (particularly ones from Latin) have retained their original plurals for some time after they are introduced. Other nouns have become Anglicized, taking on the normal "s" ending. In some cases, both forms are still competing. The choice of a form can often depend on context: for a linguist, the plural of appendix is appendices (following the original language); for physicians, however, the plural of appendix is appendixes. Likewise, a radio or radar engineer works with "antennas", but an entomologist deals with "antennae".
The choice of form can also depend on the level of discourse: traditional Latin plurals are found more often in academic and scientific contexts, whereas in daily speech the Anglicized forms are more common. In the following table, the Latin plurals are listed, together with the Anglicized forms when these are more common.
See wiki link for list of forms, which includes the plural form for criterion.
As an example, some of the Greek words ending in -a like stigma, dogma and schema, their plural ends with -ata because they are considered neutral in gender. The only time the plural is different is in the genative case (των στομάτων), where -ata is replaced by -aton.
As for criterion (κριτήριο), it too is a Greek noun which is neutral in gender. The rules governing this, however, are slightly different because it ends in -o. The plural for this kind of noun ends with -a. Therefore, το κριτήριο becomes τα κριτήρια. Again, the only time the plural is different is in the genative case (των κριτήριων), where -a is replaced with -on (not to be confused with the singular case, though).