I forget who said, "All words are infinitely polysemous," but it's probably also true that 'No words are truly synonymous.' Words will have some senses that largely overlap with those of other words, but not totally overlap. They will usually also have distinctly different senses. And when we get on to connotations, the same word will conjure up different nuances in different people. Probably, no two words are totally interchangeable (ignoring variants like artefact / artifact).
Coincidentally, I said to someone the other day, "I must be careful not to waltz off with the key." I then thought of possible substitutes for waltz in the pretty transparent idiom, and decided that I couldn't immediately think of any that are actually used. (Though apparently, the shuffle is a type of dance.) There are some amusing candidates - tango, foxtrot, conga, jive, passacaglia, slosh....
John Lawler has added: Language doesn't have any use for two words that are exactly the same; there's always contexts where you use one and not the other for some particular effect.
[2024 edit]:
It is instructive to look at one or two examples of where synonymy is deliberately stretched, in violation of the Gricean maxim of manner [be as clear, as brief, and as orderly as you can in what you say, and avoid obscurity and ambiguity].
These are deliberate attempts to misdirect.
The cryptic crossword compiler delights to use secondary (with regard to idiomaticity) or tertiary senses of a word to mislead the solver. Thus 'elementary prognostication' [7, 8] turns out to be 'weather forecast', quirkily using a far-from-default sense of 'elementary', not at all synonymous with basic, rudimentary; simple, not difficult. It is a judgement call as to where the dividing line between 'excellent encryption' and 'that sense was obsolete before Shakespeare' lies.
The obscurantism of the politician or salesman ('weasel-words'). This is either
(a) pre-emptive and hence premeditated (
'Elboneta Mini Wheats can improve children's attentiveness by 20%'
'Boost for North Elbonia as we aim to build 1,200 new homes'),
pseudo-promises which can be denied as being binding later. Or
(b) salvage attempts
- 'If you look at the definitions of 'pragmatic', you will find that one is 'relating to affairs of state', so the plan I announced was indeed pragmatic'.
I can't locate the quote, but I distinctly remember one critic saying 'X has had to really plumb the depths of his political thesaurus to come up with that justification.'
We will eat outside, weather allowing.
is not correct though it sounds ok, rather it should beWe will eat outside, weather permitting.
becauseallowing
does not collocate in this sentence.