A common construction is "Yes, that's right." Is that acceptable, or should a semicolon (or period) replace the comma? If acceptable, how can it be justified grammatically?
A comma is preferable to a semicolon in cases where the clause is a word or two. Example:
I came, I saw, I conquered.
As to the choice between a comma, a period, and an exclamation point, that would be an attempt to model the stress and emphasis with which the statement was made.
It is correct to say, "Yes, that's right."
This is a particle, which means that you don't inflect it and it's grammatically independent from the rest of the sentence. Interjections like "yes" are an example of this kind of particle.
You can put commas around these and that will be enough because of their independence.
Other examples: no wow ouch goodbye hello