The etymology and forms listed in OED indicate that this is an eye-rhymeeye-rhyme (where words look alike rather than actually sounding alike).
Symmetry has an etymology from French:
< Middle French symmetrie (1529), French symétrie (= Italian simm- , Spanish sim- , Portuguese symetria)
or possibly Late Latin symmetria. However, the 1529 entry is close to when it's first shown in English:
1563 J. Shute First Groundes Archit. sig. Aiiiv, Concerning ye proportion and simetry to vse the accustomed terme of the arte of the fornamed columbes.
OED gives various forms from the 1500s and 1600s, in addition to symmetry:
Also 15 symmetrye, simetrie, 15–16 simetry, sym(m)etrie, 16 simmetry, simmetrie, symetry.
The continued use of -ie from the French -ie would tend to suggest a pronunciation similar to French, which is highly unlikely to rhyme with eye.