refractory (adj.):

> - "stubborn, obstinate, perverse," 1610s (earlier refractorious, 1550s, refractary, c. 1600), ***from Latin refractarius "obstinate, stubborn," from past participle stem of refringere (see refraction).***

(Etymonline)

According to the following source [refractory][1], derives its meaning of stubborn, obstinate from the idea of breaking up all attempts to obey in an obstinate way:

> - (Of arguments, the force of which cannot be broken back ; of irresistible or incontrovertible arguments.) Given -to-break-back (refractory.) Abstr. subst. from refractory (refractoriness.) What does refractory means? It is used of one who breaks back all attempts to make him obey, obstinately diobedient.

From : Spelling Turned Etymology 
  [1]: https://books.google.it/books?id=0vUDAAAAQAAJ&pg=RA1-PA99&lpg=RA1-PA99&dq=refractory%20etymology&source=bl&ots=O2c5TDQsKm&sig=PqmxBeWGVKYEFlqXlgZPINntqx0&hl=it&sa=X&ved=0ahUKEwj855H32PvVAhVGOhQKHffmBGs4ChDoAQgbMAA#v=onepage&q=refractory%20etymology&f=false