MacMillan dictionary calls 'back to front' a British adverb.
Longman Dictionary of Contemporary English calls the term British English.
This accords with my experience that the phrase back-to-front, referring to wearing something backwards, is not general colloquial American usage. We do use back to front as in they seat the airplane from back to front, but that's not the same as 'backwards', which is the American English terminology. Also there may be some medical or specialty vocabulary that use the phrase back-to-front.
Check out this Word Reference thread called "How do you wear a baseball cap?" The opening post is by Wishfull, apparently a native Japanese speaker, who asks
How do natives express wearing a baseball cap forward or backward?
(There's also some verbiage about wearing a cap backwards as being bad manners, which doesn't concern us.)
Thomas Tompion, whose profile states his native language is English - England proffers
Uncle Bob (British-English)
and offers also 'the right/wrong way round'.
cuchuflete (American English), about wearing it the wrong way says
She's wearing a baseball hat with the visor facing backwards.
Then Thomas Tompion (UK) reveals that his knowledge of the game of baseball is limited as he asks
Is it only the pitcher who wears the hat back-to-front in professional matches, Cuchu?"
panjandrum (English-Ireland) in the meantime comments:
He's wearing his baseball cap back to front.
He's wearing his baseball cap the wrong way round.
Cuchuflete (who is from Maine, USA) answers Thomas Tompion with
I confess that I have no idea what you mean by back-to-front.
Myridon, from Texas, USA says:
The catcher wears his baseball cap backwards.
and uses backwards two other times.
Packard (English-USA), talking about a guy who wears his cap "with the brim to the rear," comments
Each morning I greet him the same way,"Still got your hat on backwards, eh?"
Jill-Eliza, from Portland, Oregon, USA says
In America, we don't say "the right/wrong way round." That sound British. We might say, "The right/wrong way around."
We don't say "back-to-front" where I am from either.
"She wears her baseball cap backwards." Is how I would describe the visor being in the back of her head.
In summary, each UK person in the conversation uses back-to-front. But we have an American from Maine (east coast) and an American from Oregon (west coast), on a forum discussing language usage, one who doesn't know what back-to-front means and one who states
Americans don't use the phrase. Every American response contains backwards.
As an American who's lived in several parts of the country, I concur with the Oregonian: we don't use back-to-front.
As Edwin has pointed out, Merriam-Webster has an entry for back-to-front:
of a piece of clothing
:with the back where the front should be He accidentally put the sweater on back to front.
It also marks the expression's 'popularity' as 'Bottom 10% of words'.
Regarding general American English usage, I would say 1%.
Collins dictionary says "mainly British" and "in AM usually backward".
When I search Google books for "baseball cap back-to-front" I do find some literary or prose uses. M-W might be reflecting that. Doing a general Google search returns no genuine American usage on the first two or three pages: all are from the UK or are false returns.