According to the Jargon File:
[very common; back-formation from crufty]
n. An unpleasant substance. The dust that gathers under your bed is cruft; the TMRC Dictionary correctly noted that attacking it with a broom only produces more.
n. The results of shoddy construction.
vt. [from hand cruft, pun on ‘hand craft’] To write assembler code for something normally (and better) done by a compiler (see hand-hacking).
n. Excess; superfluous junk; used esp. of redundant or superseded code.
[University of Wisconsin] n. Cruft is to hackers as gaggle is to geese; that is, at UW one properly says “a cruft of hackers”.
And for crufty:
crufty: /kruhf�tee/, adj.
[very common; origin unknown; poss. from ‘crusty’ or ‘cruddy’]
Poorly built, possibly over-complex. The canonical example is “This is standard old crufty DEC software”. In fact, one fanciful theory of the origin of crufty holds that was originally a mutation of ‘crusty’ applied to DEC software so old that the ‘s’ characters were tall and skinny, looking more like ‘f’ characters.
Unpleasant, especially to the touch, often with encrusted junk. Like spilled coffee smeared with peanut butter and catsup.
(sometimes spelled cruftie) n. A small crufty object (see frob); often one that doesn't fit well into the scheme of things. “A LISP property list is a good place to store crufties (or, collectively, random cruft).”
This term is one of the oldest in the jargon and no one is sure of its etymology, but it is suggestive that there is a Cruft Hall at Harvard University which is part of the old physics building; it's said to have been the physics department's radar lab during WWII. To this day (early 1993) the windows appear to be full of random techno-junk. MIT or Lincoln Labs people may well have coined the term as a knock on the competition.
Tech Model Railroad Club Dictionary
The quoted TMRC Dictionary was compiled by members of the Tech Model Railroad Club at MIT (who may also be responsible for foo and bar), many of worked on early programming languages and computers such as LISP and the PDP-1 and Project MAC from 1964 onwards.
The terms appear in the first edition of the TMRC dictionary of 1959, compiled by Pete Sampson who later added the italic notes:
CRUD: cruft (q.v.).
The gag here is to define the obvious word in terms of the more obscure.
CRUFT: that which magically amounds in the Clubroom just before you walk
in to clean up. In other words, rubbage.
The word was in use at the club when I wrote this definition. The sense is of detritus, that which needs to be swept up and thrown out. The dictionary has no definition for "crufty," a word I didn't hear until some years later. Rubbage is a rare term for rubbish, but I had heard it used growing up in New England.
Oxford English Dictionary
Cruft now appears in the OED, with this etymology:
Origin unknown. Perhaps an expressive formation, perhaps recalling fluff n.1, scruffy adj., crust n. However, with crufty adj. perhaps compare Jamaican English crufty coarse-looking (1943), cruffy scurfy, rough (1868), cruff (adjective) scabby, scurfy, naturally rough, coarse, uncooth (c1915), all ultimately related to scruff n.1: see F. G. Cassidy & R. B. LePage Dict. Jamaican English (1967) at cited words.
A number of other etymologies have also been suggested.
Anything unnecessary or redundant; esp. poorly designed or unnecessarily complex computer software, e.g. that containing sections of obsolete code. Also: filth, esp. that which builds up over time; detritus.
Their first quotation is the 1959 TMRC dictionary.
Nowadays, it is mainly used in terms of redundant, poor quality or wasteful source code that should best be removed; but can also be that unpleasant-to-touch dirt that accumulates on keyboards and inside computers.