We can go a bit further than your ngram. Here I use a search for (worth _DET_ millions + worth _NUM_ millions):eng_us_2012,(worth _DET_ million + worth _NUM_ million):eng_us_2012,(worth _DET_ millions + worth _NUM_ millions):eng_gb_2012,(worth _DET_ million + worth _NUM_ million):eng_gb_2012 to compare with different determiners, articles and numerals, and to do so separately for British and American sources:
As we can see, it shows the singular form to be more popular, and increasingly so.
There are some serious problems with this analyses though. First, it would include phrases like "five million dollars", "five million roubles", "five million pounds" etc. We can reasonably expect this to be more common than "five millions dollars" etc. though we do find "The family were content, for it was known that I was sole heir to an uncle held to be worth five millions of dollars." from Mark Twain's "The Canvasser's Tale" accounting for quite a few hits.
We could try subtracting worth _DET_ million _NOUN_ etc, but that pushes past the limits of ngrams. So instead we can try (_DET_ millions + _NUM_ millions - _DET_ millions _NOUN_ - _NUM_ millions _NOUN_),(_DET_ million + _NUM_ million - _DET_ million _NOUN_ - _NUM_ million _NOUN_). This brings in other phrases beyond the "worth..." phrase, but in combination with the above is worth looking at:
It remains, that we are unfairly comparing "one million" and "a million" with "one millions" and "a millions", where the latter would only happen in error. I do think, that even considering that and the caveats that must apply to ngrams, that we can consider that the form "worth X millions" has genuinely decreased to become a rarity.
But why?
It's obvious that both "worth 20 millions" (as you found) and "worth 20 million" (as you would say you would expect) mean what could be expressed (assuming US currency) as either "worth 20 million dollars" or "worth 20 millions of dollars". Million is being used as a shorthand with what is being denominated elided.
As such, it is being used as a noun in its own right. We still do this, we have millions, thousands, hundreds, dozens, tens. We even have fives, sixteens etc. but generally only when items come in groups of that number.
But, as we still have that usage, and as we can find (if we look into the actual texts that mention "worth five millions" and so on) that it was used regularly enough, it would simply seem that of two choices available to writers, one became idiomatic, and that left the other to die out.
A factor could just be that we less often considered a "million" as a "thing" in its own right.
From the 19th well into the 20th Century, million in terms of total assets was the denominator of serious wealth, replacing the use that lets us know Jane Austen's Mr. Darcy is seriously wealthy because he has "ten thousand a year".
Now to be seriously wealthy, you would need several millions (I think Darcy is estimated to be worth around 3million sterling, total assets, in today's money), and to be notoriously wealthy you would need billions.
I'd offer that a factor in this is simply that dealing with such sums is more usual to us. Aside from personal wealth, the figures we encounter in terms of reports of big business, national economies, and so on throw millions around all the time. We also have more non-financial encounters with millions in terms of popular science.
In all, there was a mythos to a million, that it doesn't have today. Where once 20 millions would be 20 amazing things, now it's a number.
But I think that was only one factor, if it was one at all. When idioms develop, they push out alternatives like cuckoos in a nest. When we become used to "worth 20 million", the alternative seems strange.
