It is usual to capitalise only the first part of a compound or hyphenated word in a title (though both parts in a headline) but there are common exceptions, so be guided by the relevant Style Manual if there is one. Specific organisations will have their own house rules regarding capitalisation.
For example, The Chicago Manual of Style (section 7.128) advises that if the first element is a prefix, then the second element is not capitalised unless it is a proper noun or adjective, even in a title.
From the English Language Institute at the University of Delaware:
"You capitalize the first word in a hyphenated word within a title. Capitalize the second word only if it is a noun or an adjective that is as important as the first word."
That is, they advise you do capitalise the second word in some cases. In the examples you have given, I would write:
Stripped-down Strap-on Make-up (wow, sounds pretty wild, what are you writing about?!)
The Australian (Government) Style Manual also says:
"Use title case for all section headings, figure and table captions. Title case capitalises every main word, but not the ‘little’ words such as on, by, for, of, the, an, a, for, etc. Do not capitalise the second word in a hyphenated word."
Commonly, articles, prepositions, coordinating conjunctions, or musical modifiers will not be capitalised, as in:
Hit-and-Run Driving Statistics
I'd advise that you capitalise everything in a headline, but for a title, just go with what looks right unless you are writing for an organisation with a specific house style.