Google Books examples of 'eightteen'
A Google Books search turns up a boatload of instances where eighteen is spelled eightteen, but the majority of them appear to be straight-up variant spellings, with no line break involved. Still, in a considerable number of instances—probably too many to be merely coincidental— eightteen appears at a line break and is broken between the double t's. Many of these are from the eighteenth and nineteenth centuries. For example, from R. Denson, A New Travellers Companion Through the Netherlands (1754):
From Delft to the Hague a boat goes of eight-teen times every day in the morning from 7 'clock, till half hour past 7. at night almost every half hour the freight is 2. pence half penny.
From"An Officer," The Gentleman's Guide in His Tour Through France (1783):
I must beg leave to observe, it is a general conceived notion in England, that is necessary to have a considerable fortune to make the tour of France: so it is, I confess, if a man is determined to be a dupe to Frenchmen, and enter into all the follies, vices, and fopperies, of that vain, superficial people; but I can with veracity declare, that during eight-teen months I was abroad, it did not cost me 150l. sterling.
From George Taylor, A History of the Rise, Progress, and Suppression of the Rebellion in the County of Wexford in the year 1798 (1829):
The third time they took out eight-teen, and were massacring them, when Dick Monk rode into town from Vinegar-hill, with 'his shoes and Stockings off, and shouting, “ D—n' your souls you vagabonds, why dont you go out and meet the enemy that are coming in, and not be murdering thus in cold blood?".
From Samuel Cooper, A Dictionary of Practical Surgery, seventh edition (1838):
Mr. Saunders found, that the greatest success attended the operation between the ages of eight-teen months and four years.
From "The New John Gilpin," in Punch (1846):
Now RICHARD COBDEN said to him,/"Protected Corn has been/Thro' thrice ten tdious years, since eight-/Teen hundred and fifteen.
And more recently, from Farley Mowat, The Farfarers: A New History of North America (2011):
Now I found myself on the verge of a depression whose size and shape were a good math for most of the boat-roofed house foundations scattered across the eastern Canadian Arctic. Roughly fifty feet long (nobody had a tape measure, so we had to pace it) it was perhaps eight-teen wide.
And from Gregory Miller, Daddy & Daddy Against the Odds (2012)
They had a pool in the back yard and a patio covered with grapevines and a Botchy Court. Our bedroom was about eight-teen feet long by ten feet wide. The living room was just as big.
In contrast, we have this early example from Adam Smith, An Inquiry Into the Nature and Causes of the Wealth of Nations (1784):
The rise in the price of porter, occasioned by an additional tax of three shillings upon the barrel of strong beer, has not raised the wages of common labour in London. These were about eigh-teen-pence and twenty-pence a day before the tax, and they are not more now.
Assessment of the Google Books results
In significantly more than half of the 70-odd Google Books matches for "eightteen" that I spot-checked, the word appeared as a chapter number or on a line of text where it was not severed by a line break. Evidently, in these instances, it was either a variant spelling or a typographical error. But especially between the 1700s and the very early 1900s, eightteen appeared often enough at line breaks to suggest that there may have been some intention in at least some cases to duplicate the t because of where the word fell on the line.
It is important to understand the relatively low frequency of these occurrences, which may be illustrated by an Ngram chart mapping eightteen (blue line) versus eighteen (red line) across the period 1700–2005:
Throughout the period, the incidence of eightteen is so low in comparison to the frequency of eighteen that the line for it looks completely flat. A look at the frequency of eightteen alone across the same period shows that the word's period of greatest frequency was between 1750 and 1900 (as the individual Google Books results indicate, too):
Looking at the individual instances of eightteen in the Google Books results, I was struck by how rarely the matches included a discussion of the spelling. The two exceptions from the period between 1900 and 2015 appear in Greg Brooks, Dictionary of the British English Spelling System (2015), cited in the OP's question, and in Henry Shaw & Richard Dodge, The Shorter Handbook of College Composition (1965) [combined snippets]:
When two words [or a prefix and a word, or a word and a suffix] are combined, the first ending with the same letter with which the second begins, be sure that both letters are included.
accidentally irresponsible really bathhouse meanness roommate bookkeeping misspelling suddenness cruelly occasionally unnecessary dissatisfied overrated unnoticed drunkenness override withholding
The only important exception to this rule is eighteen, which, of course, is not spelled eightteen.
It thus appears that the recommended rule that the OP finds in Don Knuth, The TeXbook (1984) is not one supported in the past 100 years by any other book in the Google Books database. I might add that my electronic searches of that book for the words "eightteen," "eight-teen," and "eight- teen" all came up empty.
Whether some eighteenth- or nineteenth-century publishers consciously added a second t to eighteenth for aesthetic reasons when the word broke at a line break is unclear (though not impossible). But it seems unlikely that any such rule has held sway at any major English-language publisher in the past century.