Speaking very technically, to a typesetter that’s not a hyphen — it’s a dash. One possible easy-to-remember way to keep them straight is
“Hyphens join. Dashes separate.”
The full stop (period), the exclamation point, and the question mark are used to end a sentence (independent clause) without committing to anything following them. In contrast, the semicolon, occasionally the ellipsis or the colon, and the dash (can) all connect one logical sentence to another in some fashion.
These sentence-joiners (meaning
—) always mean something a bit different than using sentence-terminators (meaning
!) and just going on to have a new sentence in the regular way. Indeed using anything other than a period to end a sentence with means something special, the joiners just as much as the other terminators.
Of the dash, the OED says:
A horizontal stroke of varying length (-, —, ——) used in writing or printing to mark a pause or break in a sentence, a parenthetic clause, an omission of words or letters or of the intermediate terms of a series, to separate distinct portions of matter, or for other purposes; sometimes implying the use of strong language; hence as a mild substitute for devil.
Using abnormal punctuation at a high frequency can make your text look informal or non-standard. The careful writer chooses their punctuation with care. These all have their places. The Wikipedia article I initially linked to above has some suggested guidelines that may help you.