I don't think tails has anything directly to do with what is on the other side of the coin, but rather it is an expression of opposites: the head is at one end of spinal column, the tail at the other (think 'dog' nose to tail are opposites, rather than head and feet). The expression can't make head nor tail of it expresses this concept of opposites, and may be where heads or tails comes from.
The first recorded use of "tails" to mean the reverse side of a coin occurred in a 1684 comedy, "The Atheist," by playwright Thomas Otway. A character in the play advises someone, "As Boys do with their Farthings ... go to Heads or Tails for 'em."
As far as the coin toss goes, it is far from recent. Cross and Pilepile was played in England for many centuries. The cross was the major design element on one side of many coins, and the Pilepile was the bottom part of the die used to cast the other'cross' side of the coin. Brewer's Dictionary of Phrase and Fable (1898) Samuel Butler used the phrase in the 1600s: “Whacum had neither cross nor pile.” (Butler: Hudibras, part ii. 3.)
Before that, it was done by the Romans, and was called navia aut caput ("ship or head"), as some coins had a ship on one side and the head of the emperor on the other.