In American English, commas and periods are placed inside the quotation marks. Therefore, the following one is how the sentence is written in American English.
A common proverb is, "A rolling stone gathers no moss."
The period is placed inside the quotation marks even if it's not part of the quoted phrase, and it is written only once. The period is placed inside the quotation mark even if the part between the quotation marks is not a complete sentence.
The word "punctuation" derives from two ancient roots: "punc," meaning "a hoodlum," and "tuation," meaning "desire to become."
(The etymology is false; it's just a funny example taken from Comma sense: a fun-damental guide to punctuation, written by Richard Lederer and John Shore.)
Just to complete the information, these are other examples of punctuation used with quotation marks.
I confessed, "I love punctuation"; then I explained "I also love spelling rules."
"Punctuation": Ah, how that word makes my heart tump.
I asked, "Do you think I love punctuation?" How wonderful that I can finally confess "I love punctuation"!
As far as I know, in British English the sentence is written as follows:
A common proverb is "A rolling stone gathers no moss".