It seems to me that semicolons are rarely used today in ordinary English writing - even in newspapers and books. They appear to have been replaced, in many cases, by em-dashes and hyphens (the hyphen being used on the Internet and/or in informal communications). In some cases, semicolons have been replaced by periods (i.e. something which might have previously been one sentence would now be written as more than one sentence). Over what period of time did this change take place, and why?
The short answer, I think, is that the em-dash makes it easier to read the sentence and thus more likely that the reader will understand your point and want to continue reading.
Readers today are inundated with far more to read than ever before, and any structure that forces the reader to do more work will be detrimental to their likelihood of reading further.
Grammatically, semicolons are used to connect two independent clauses without the use of a linking preposition. However, the type of connection is not stated outright; the reader is supposed to assume a reason for not separating the clauses into sentences. (In the last sentence, the implied connection between clauses could be expressed by the word therefore.)
When a writer uses an em-dash rather than a semicolon, there is white space around the connection. White space, particularly for visual learners, signals that there is some work to be done by the reader to understand the connection between two things. It also allows metaphorical space for that to happen, since it forces the reader to pause and make the connection.
A semicolon doesn't offer that same amount of white space. It is a subtle cue, and can slow the reader down if they don't see the period above the comma. (Anything that makes a reader pause and say "huh" will increase the likelihood that they will not continue to read.)
It fits in with a longer-term trend toward shorter sentences and plainer writing. Semicolons are only useful in long sentences.
I am not sure why this happened. On the one hand, as Martha F.’s answer indicates, good writers these days are keenly aware that
they are competing with Angry Birds for the reader’s time and interest; and
the reader very likely isn’t really concentrating,
and perhaps it is as a consequence that sentences like this one, from A Tale of Two Cities:
Daring burglaries by armed men, and highway robberies, took place in the capital itself every night; families were publicly cautioned not to go out of town without removing their furniture to upholsterers’ warehouses for security; the highwayman in the dark was a City tradesman in the light, and, being recognised and challenged by his fellow-tradesman whom he stopped in his character of “the Captain,” gallantly shot him through the head and rode away; the mail was waylaid by seven robbers, and the guard shot three dead, and then got shot dead himself by the other four, “in consequence of the failure of his ammunition:” after which the mail was robbed in peace; that magnificent potentate, the Lord Mayor of London, was made to stand and deliver on Turnham Green, by one highwayman, who despoiled the illustrious creature in sight of all his retinue; prisoners in London gaols fought battles with their turnkeys, and the majesty of the law fired blunderbusses in among them, loaded with rounds of shot and ball; thieves snipped off diamond crosses from the necks of noble lords at Court drawing-rooms; musketeers went into St. Giles’s, to search for contraband goods, and the mob fired on the musketeers, and the musketeers fired on the mob, and nobody thought any of these occurrences much out of the common way.
are now rarely found in nature.
But I think that view is not quite adequate. There’s really nothing particularly taxing about this thirty-clause pile-up, no reason you couldn’t stop halfway through, set it down, and read the rest tomorrow. It’s just that it would be so much more manly and Hemingway-ian with full stops. A matter of changing tastes.
I'm going to go out on a limb and suggest, on no more evidence than my own inattentive impression, that @Billare is right: your premise is wrong.
Pages 143 ff. of this show that dashes, ellipses and the like were more common in the past than you might think. But they showed up most often in relatively ephemeral works—those which are least likely to come to modern attention. The same is true today: dashes and short sentences are characteristic of informal writing.
It is true that "formal" writing is somewhat more casual now than it used to be; this has always been true, in every age. But seems to me that most of the difference you observe reflects the facts that a) more informal and downright casual literature is available to your inspection than ever before, and b)it is not clearly distinguished by medium from formal writing. Works of philosophy, scholarship, science, and 'high' literature come to you now from the same source and in the same format as news, chat, personal correspondence, and light or sensational reading.
So you're now averaging "formal" and "ephemeral" together, which artificially diminishes the relative frequency of characteristically "formal" punctuation.