Although the phrase became something of a Hollywood cliché in the 1930's, it was around long before that and didn't die out until the civil rights movement of the 1960's.
According to a couple of sources the phrase appeared around 1828 as a description of who should be allowed to vote. Prior to 1790, one had to own land and/or meet other additional qualifications in order to vote (actual qualifications were set on a state-by-state basis), but those restrictions were gradually being dropped. With the election of Andrew Jackson as President of the United States in 1828, Jackson's position that property ownership restrictions be abolished everywhere apparently gave rise to the phrase that all you should need to be in order to vote is "free, white, and 21". (Being male was still required, but so obvious it did not need to be stated.) While some states had gotten there before 1828, it was not until 1850 that these other restrictions were finally eliminated across the entire country. This also appears to be the point of the quote from South Carolina Bar Association that FumbleFingers dug up: "free, white, and 21" used to (1850-1870) cover everyone who was allowed to vote, but with black men being allowed to vote, this was not the case anymore.
In this context, "free" referred not to not being a slave but primarily to not being in jail or prison for committing a crime. Even today, most people incarcerated for crimes cannot vote. Prior to the 15th amendment to the constitution in 1870, black men could not vote in most states whether they were free or not.
Of course "free, white, and 21" eventually came simply to mean unencumbered (by law or custom), and women would use it well before they would be allowed to vote to assert their right to make other choices in their lives, such as if and when and to whom to get married (or otherwise get sexually involved with). This was usually how it showed up in the Hollywood movies of the 1930's.