Physic and physical are obviously related. Both have to do with medicine and the natural world from the fourteenth century onward. The Middle English Dictionary entires for physik and phisical highlights these overlaps with quotations that show that medical practice was already closely associated with the body (minimal translations in parentheses):
a1500(a1450) Ashmole SSecr.(Ashm 396)20/3 : He made this boke..techyng..philosophik and phisik doctrine, pertenyng vnto lordes for kepyng of the helth of their bodies. (He made this book teaching philosophic and physic doctrine, pertaining onto lords for keeping the health of their bodies.)
c1450 Lydg.SSecr.Ctn.(Sln 2464)1803 : Sleep, receyved in tyme and mesure..ffroom these seknessys the boody doth Recure, Which previd is by phisichal prudence.
(Sleep, received in time and measure ... from these sicknesses the body recovers, which is proved by physical prudence.)
So between this point and 1780, physical (but not physic) grew to relate specifically to corporeal bodies even independent of the medicine that treats such bodies. The OED gives a clue for how physical could drift to meaning bodily in one of its quotations:
IV. Senses relating to the human body.
a. Of or relating to the body; bodily (as distinct from mental); corporeal.
1737 (title) Onania: or, the heinous sin of self-pollution, and all its frightful consequences... With spiritual and physical Advice to those who have already injur'd themselves by this abominable Practice.
Physical would often appear in contrast to spiritual and related words in texts throughout that century, until physical was acting to describe a corporeal body or desire:
1780 J. Bentham Introd. Princ. Morals & Legisl. (1789) xiv. §3 Suppose for example the physical desire has for its object the satisfying of hunger.
1778 J. R. Forster Observ. Voy. round World iv. 154 The women..are deprived in their matrimony of that share of physical love which, in a monogamous condition, would all be theirs.