As Josh61's answer notes, one apt slang term is bluenose. Wentworth & Flexner, Dictionary of American Slang (1960) has this entry:
blue nose n. A person with strongly puritanical moral convictions; one who believes that having a good time is immoral; an ultraconservative. 1956: "It was 1917 and America was at war and the moral bluenoses were sniffing around the army camps and keeping our boys pure, so they could make the world safe for democracy." S. Longstreet, The Real Jazz Old and New, 61. Orig. in Colonial times = an aristocrat.
Chapman & Kipfer, Dictionary of American Slang, Third Edition (1995) offers two definitions of the term:
bluenose n 1920s A prude; prig; self-appointed moral arbiter ...
Bluenose n by 1830s A native of Nova Scotia, esp. a Maritimes fisheman [fr the color of a very cold nose]
J. E. Lighter, Random House Dictionary of American Slang (1994) has a very lengthy entry for bluenose, which it defines as meaning "a native of Nova Scotia [or later of any Maritimes province in Canada]," "a New Englander," or "an excessively puritanical person; prude." This dictionary includes a citation from 1992 in the New York Observer, indicating that the usage remains current:
In the screening room are...the head censor...and Tyler, an apprentice blue-nose.
As these definitions suggest (and as Josh61 again notes), the term puritan may be apt, as well, as in "He's such a puritan [or "so puritanical"] about public displays of affection."