Are synonyms exact? This is an excellent question, and it gets to the concept of how meaning is constructed in language.
With your example from arithmetic, the thing the symbol 5 refers to is fixed. Mathematicians have a series of absolute rules for how you can use it and what it is equal to.
Language, and in particular, the relationship between a thing in a language (e.g., the word happy or phrase a happy occasion) and the meaning of that thing, is not fixed. Many linguists and philosophers would say the meaning of the set of symbols or sounds "happy" is constructed based on how it is used.
You can think of an individual's understanding of a word -- you may look up a word in the dictionary and see a string of other words that describe a meaning, and you may also encounter that word in text and speech. All of these things help you understand what you and others believe the word to mean.
You can also refer to the way dictionaries generate their definitions. Merriam-Webster describes this process in an answer to the faq "how does a word get into the dictionary". The dictionary meanings themselves are based on the way the words are used.
If meaning comes from usage, since two synonyms will occur in different texts, with different contexts, they may have similar or very similar meanings, but they can't have exactly the same meaning.
You may make an argument that in a specific utterance you could replace one word for another without a change in meaning, that the thing either of those words referred to would be the same. However, once that utterance occurs, only one of the possible words was used, and both meanings are changed either by being selected or not.