Nummy as a baby-talk version of yummy predates Sesame Street, probably by some decades. The New Oxford American Dictionary (2001) puts its origin (as a variant of yummy) in the early twentieth century.
I found two possible (though not entirely unambiguous) instances of nummy for yummy in a Google Books search from the years 1915 through 1921. From Lucy O'Connor, Mary's Meadow Papers (1915):
…and they carry little Dossie, sucking her "nummy," and say "It doesn't matter bringing Dossie, because she is too young to eat anything"; and they know I shan't mind having Rosie Matthews, because her mother is ill ; and they think we'll like poor Katie Tipton, because she's fame and walks with a crutch; and Gerty Collier is staying with the Ridlers, and goes back home to London on Monday, and she wanted to come because she didn't even know what a tent-party was.
From John Galsworthy, "A Stoic" (1918):
There was also a little pink note with one blue forget-me-not printed at the top. It ran: "DEAREST GUARDY,—I'm sorry this is such a mangy little valentine; I couldn't go out to get it because I've got a beastly cold, so I asked Jock, and the pig bought this. The satin is simply scrumptious. If you don't come and see me in it sometime soon, I shall come and show it to you. I wish I had a moustache, because my top lip feels just like a matchbox, but it's rather ripping to have breakfast in bed. Mr. Pillin's taking us to the theatre the day after to-morrow evening. Isn't it nummy! I'm going to have rum and honey for my cold. Good-bye, Your PHYLLIS."
The first unmistakable instance, however, is from an unidentified 1964 article by Charles Morris, quoted in Richard Fiordo, Charles Morris and the Criticism of Discourse (1977):
...the interpretant would be a disposition to act toward a designated object as if it would be satisfying or unsatisfying. Thus if a mother tries to get her child to swallow a teaspoonful of castor oil by saying "nummy num", the child is set for something he will favor. Since he does not like it when he tastes it, and if the mother continues to talk like this in a variety of situations, the term 'nummy num' will change from a positive appraisive sign to a negative appraisive sign...
The main complication with the early examples of nummy is that several nineteenth-century sources use nummy to refer to numbness or as slang for numbskull.
Thus, R. Oliver Heslop, "Dialect Speech in Northumberland," in Lectures on Northumbrian History Literature and Art (1898) reports:
But he [the goodman of Northumberland] would liever not send the scrat of a pen to the maister: his hand was so stiff and nummy.
And Heslop again, in Northumberland Words (1893–1894), includes this entry:
NUMBY, NUMMY, a numskull. "Co'by! ye numby."
Further, as noted "Exclamations in American English" in Dialect Notes (1924), the phrase "num, num, nummy, num" can signify "humming or joy," as indeed happens in this episode from Zona Gale, Miss Lulu Bett (1921):
They ate, in the indecent silence of first savoring food. A delicate crunching of crust, an odour of baked potato shells, the slip and touch of the silver. "Num, num, nummy-num!" sang the child Monona loudly, and was hushed by both parents in simultaneous exclamation which rivaled this lyric outburst. They were alone at table.