The phrase "wash hands of" is supposed to mean giving up or having nothing to do with something.
But why is it "of" and not "off"? Using "off" sounds more like giving it up and distancing oneself from the object/subject.
In the words of the Oxford English Dictionary,
Of represents what was in earlier stages of the language a genitive. Other verbs which have this feature include cheat, defraud, accuse, suspect and avail.
The significance of the preposition of is brought out well in the definition here:
[emphasis on of is mine]