I haven't found any common factor distinguishing the elements with names ending in -ium from the elements with names ending in another letter followed by -um. It seems fairly arbitrary, and in fact there was some variation historically between forms ending in -ium and -um for tantalum and lanthanum (and there is still variation between aluminium and aluminum in American English).
Most elements are metals. Many of the nonmetals are halogens or noble gases, which have different naming patterns using the suffixes -ine and -on respectively. But the nonmetals helium and selenium, and the metalloids tellurium and germanium have the -ium suffix also. The user J... left a comment saying that helium was originally thought to be a metal; a quote supporting this can be found in Linear Christmas's answer to the related Chemistry SE question Why do the names of most chemical elements end with -um or -ium?. The Oxford English Dictionary (OED) entry for selenium says that it and tellurium were "formerly classed among the metals".
Apparently, in Latin the names of metals always had the neuter grammatical gender ( Lectures on Syntax: With Special Reference to Greek, Latin, and Germanic, by Jacob Wackernagel, edited by David Langslow, published 2009). Examples of this are ferrum "iron," aurum "gold," cassiterum "tin" (a loanword from Greek κασσίτερος). Latin neuter nouns of the second declension regularly take the suffix -um when singular and in the nominative or accusative case. (Janus Bahs Jacquet has pointed out in a comment that some metal names in Latin belonged to other declensions and so did not take this suffix, such as aes "copper/bronze/brass." But I have the impression that new words formed in scientific Latin are generally in the first or second declension.)
So the -um part of the suffix -ium is just this second-declension neuter inflectional suffix. The suffix -i- (which comes before the inflectional suffix; words with -i- may end in -ia/-ius/-ium depending on their gender) is a common derivational suffix in Latin. Generally, it's used to derive adjectives from other nouns (Wiktionary) but in Latin adjectives and nouns can often interchange, so it's often also possible to use words with this suffix as nouns. I haven't found a reference that explains why -i- shows up in so many of these element names; I'd assume it's because as new elements were discovered people named them in honor of existing things and the easiest way to derive a new name like this in Latin is to use -i-. For example, rhodium is apparently named after roses (the Ancient Greek word for "rose" is rhodon). Also, as more and more elements were named this way it became a noticeable pattern that people might follow even if they were not aware of its origins.
The OED doesn't provide an extensive explanation of the origins of the use of -ium, but it mentions the Latin metal names and gives some historical context for the English ones:
the names of sodium, potassium, and magnesium, derived from soda,
potassa or potash, and magnesia, were given by Davy in 1807, with the derivative form -ium; and although some of the later metals have
received names in -um, the general form is in -ium [...].
As I said, I can't see any factor connecting all of molybdenum, platinum, tantalum, and lanthanum. (The names tantalum and lanthanum were coined more recently than molydenum and lanthanum, so I have re-arranged your list to put them last.)
Molybdenum, according to Wiktionary, is derived from the Greek name for lead, μόλυβδος molybdos. The OED provides some additional details about how the -en- got in there: it references the earlier word "molybdena", from Latin molybdaena, from ancient Greek μολύβδαινα molybdaina "a kind of lead ore" which according to the OED ends in "-αινα, suffix forming nouns". The rule about metal names having neuter gender in Latin does not seem to apply to the names of ores: there are also words like galena (feminine) "lead-ore; dross", anthrax (masculine) "cinnabar", and (lapis) haematites (masculine) "red iron-ore/hematite" (it also had other meanings, apparently). I don't see any particular reason for why molybdenum was used as the name of the element instead of molybdenium.
Platinum seems to be from a Spanish word, platina, meaning "little silver." I guess it makes some sense to adapt this by simply changing the gender suffix to -um rather than deriving a new word with -i-. But the process seems fairly arbitrary.
Tantalum (discovered and named in 1802) is named after Tantalus. I don't see any reason why it isn't tantalium. In fact, Wiktionary lists that as an obsolete spelling, and there is a question that was asked earlier on this site about that variant of the word: Reason why tantalium became obsolete.
Lanthanum (named sometime around 1840) is named after a Greek word meaning "to go unnoticed." I don't know why it doesn't have the suffix -ium; in this case as well, it seems completely arbitrary to me. Wiktionary lists lantanium and lanthanium as obsolete alternative forms, and there is a question about them here: Lanthanum vs lanthanium.