The word swear comes from the Old English swerian (past tense swor / pp sworen), meaning a solemn oath.
In the original use, swear means a solemn promise ("I swear that"), made to someone ("I swear to"), optionally with collateral put down in the event that the promise is broken ("I swear on"), for instance "I swear on my life", "I swear on my homestead" or "I swear on my honor".
Note that you swear to somebody. For instance, "I swear allegiance to my liege" means I promise loyalty to my lord/king.
"I swear to God" evolved from this meaning a solemn promise to God — generally as an appeal to a higher power for increasing the weight of a promise ("I swear to God that I will pay my rent by next week") or as a way of adding emphasis and solemnity when announcing a self-imposed duty e.g. "I swear to God that I will find the perpetrator of this crime".
The extended meaning (i.e. as an expletive) comes from medieval times (c. early 15 century) in relation to "I swear to God". In the event that someone broke this promise, it was interpreted by many (including the Catholic Church) as a sin, because it was a taken as a direct breach of the third commandment:
Exodus 20:7 Do not take the name of the Lord in Vain
Note that the objection here is not "taking the name of the Lord" (i.e. swearing to God), but rather taking it in Vain — i.e. swearing to God and subsequently breaking the promise.
The notion of swearing being overtly bad, or crude, evolved from this meaning — where saying "I swear to God" is a solemn oath not to be taken lightly — for fear of going to hell.
For this reason, invoking sacred names over trivial incidents was therefore something that was not to be done in polite company, and something that children should be actively discouraged by their parents from doing.
Until the mid 1700s, use of obscenities ("cursing") was treated completely separately, and during the mid 1700s the two became lumped together as "swearing and cursing". By the mid to late 1700s this was being frequently shortened to "swearing", and the term "swearing and cursing" eventually petered out in the mid 1800s.
The word swearword is an American English colloquialism from roughly 1883.
In modern English, swear meaning to promise has become increasingly uncommon except perhaps in legal contexts and particularly in American English — and consequently "I swear to God" meaning a solemn promise to God would be a very archaic interpretation. The meaning of "I swear to God" nowadays is pretty much exclusively used as an expletive.