Nummy as a baby-talk version of yummy predates Sesame Street, probably by some decades. The New Oxford American Dictionary (2001) puts its origin (as a variant of yummy) in the early twentieth century.
I found two possible (though not entirely unambiguous) instances of nummy for yummy in a Google Books search from the years 1915 through 1921. From Lucy O'Connor, Mary's Meadow Papers (1915):
…and they carry little Dossie, sucking her "nummy," and say "It doesn't matter bringing Dossie, because she is too young to eat anything"; and they know I shan't mind having Rosie Matthews, because her mother is ill ; and they think we'll like poor Katie Tipton, because she's lame and walks with a crutch; and Gerty Collier is staying with the Ridlers, and goes back home to London on Monday, and she wanted to come because she didn't even know what a tent-party was.
From John Galsworthy, "A Stoic," in Five Tales (1918):
"DEAREST GUARDY,—I'm sorry this is such a mangy little valentine; I couldn't go out to get it because I've got a beastly cold, so I asked Jock, and the pig bought this. The satin is simply scrumptious. If you don't come and see me in it sometime soon, I shall come and show it to you. I wish I had a moustache, because my top lip feels just like a matchbox, but it's rather ripping to have breakfast in bed. Mr. Pillin's taking us to the theatre the day after to-morrow evening. Isn't it nummy! I'm going to have rum and honey for my cold. Good-bye, Your PHYLLIS."
The earliest unmistakable instance that Google Books returns, however, is from Charles Morris, Signification and Significance (1964), partially reproduced in Charles Morris, "Writings on the General Theory of Signs* (1971):
In the case of appraisive signs, the interpretant would be a disposition to act toward a designated object as if it would be satisfying or unsatisfying. Thus if a mother tries to get her child to swallow a teaspoonful of castor oil by saying "nummy num", the child is set for something he will favor. Since he does not like it when he tastes it, and if the mother continues to talk like this in a variety of situations, the term 'nummy num' will change from a positive appraisive sign to a negative appraisive sign—or the child will come to regard his mother as a liar.
The main complication with the early examples of nummy is that several nineteenth-century sources use nummy to refer to numbness or as slang for numbskull.
Thus, R. Oliver Heslop, "Dialect Speech in Northumberland," in Lectures on Northumbrian History Literature and Art (1898) reports:
But he [the goodman of Northumberland] would liever not send the scrat of a pen to the maister: his hand was so stiff and nummy.
And Heslop again, in Northumberland Words (1893–1894), includes this entry:
NUMBY, NUMMY, a numskull. "Co'by! ye numby."
Further, as noted "Exclamations in American English" in Dialect Notes (1924), the phrase "num, num, nummy, num" can signify "humming or joy," as indeed happens in Zona Gale, Miss Lulu Bett (1921):
"Num, num, nummy-num!" sang the child Monona loudly, and was hushed by both parents in simultaneous exclamation which rivaled this lyric outburst. They were alone at table. ...
There was a moment's silence into which Monona injected a loud "Num, num, nummy-num," as if she were the burden of an Elizabethan lyric.
Update (May 16, 2020)
A much earlier instance of "nummy" clearly used in the sense of "yummy" appears in Nancy Wynne, "Just Gossip About People," in the [Philadelphia, Pennsylvania] Evening Public Ledger (September 20, 1918):
You never tasted anything like the waffles. Umh—umh! I want some more today. And you had to wait quite a while, too, yesterday, because the stoves were not set up in time. However, I got mine at 1 o'clock, and nummy! nummy! but they were good.
And from Henry Hanemann, "'Children Who Prefer Sweetmeats...'" in Life Magazine (December 9, 1920):
"See!" cooed the nurse. "See what nursie has for nice itty gentyman's tea. Ni-ice, twee-et, oaty mee-al."
Horace looked over at Albemarle. But that "itty gentyman" had already rushed over to the diminutive supper table, and was beating frantically upon it with his silver spoon.
"Nummy, nummy, nummy!” he clamored shockingly. "Ni-ice oaty meal. Nummy, nummy, nummy!"
"Nummy, nummy, nummy!" agreed Horace heartily, taking his place opposite his host.
So evidently the expression has been in use for more than a century among some English speakers.