Most everyone knows what the common turn of phrase all bets are off means: "anything can happen." But all idioms have to start from somewhere, and the question I'm wondering is how did this one start. Google informs me it comes from a gambling context, and I'm inclined to believe that, having seen the movie Snatch, but what I'm not understanding is how the literal meaning of "all bets are off" -- that is, the house will not be taking any bets today -- would have evolved into "anything can happen." Can someone enlighten me to a plausible pathway?
You asked for a plausible development, so here goes... note that I can't prove that this is how it developed, but I've at least tried to give a genuine usage example of each of the senses in the sequence, with dates where possible.
(i) This phrase was originally used in betting on sports, and in particular horseracing. The meaning is that not only is the bookmaker not taking any more bets, but that existing bets placed on the competition are null and void. This is usually because the race has been cancelled, but can also be due to other factors - for instance, the starting lineup changing so substantially that forcing people to honour their bets would be unfair, or in the event of an irregularity in the conduct of the race. (It can also be because the event was drawn.)
Here is a modern example in horseracing rules:
Or it can refer to a draw (1897):
Or an irregularity (1901):
Or a change in the starting lineup (1839):
Or because of misinformation about the race (1881):
I believe that the phrase became embedded in people's consciousness, since at racecourses (and other sporting events) it would commonly be announced "The 14.35 has been cancelled; all bets are off." However, given the oral nature of such announcements, it is hard to track down any evidence, so this is largely speculation, but it is still occasionally used in a manner consistent with this, for example (2009):
(i)(a) By direct association, the phrase is often used by the media with the implication that the event in question has been cancelled (2008):
(ii) From the first meaning ("bets are null and void") is derived a sense of all existing agreements are cancelled: that there is a return to the status quo ante, because even existing bets are not honoured (although the stake is returned). Examples of this usage are hard to tell apart from the meaning in the next paragraph, so I have put them all together there.
(iii) Following on from that (and no doubt assisted by the similarity of the phrase the gloves are off), the phrase is applied to situations where the circumstances have changed substantially, to the extent that former truces, alliances and restraint no longer apply.
This can mean that former commitments of non-aggression are discarded (2010):
Or an amnesty is cancelled (2009):
Or former leniency will be reversed (2011):
Or indeed that rules no longer apply (2011):
(iv) Of course, a consequence of this is that, since the rules of the game have changed, there is a sense that "anything can happen" in that what was previously unthinkable is now entirely possible. This sense, probably combined with one aspect of the original meaning (i.e. the circumstances have changed such that former bets/predictions are void), then leads, finally, to the definition quoted in the question - basically the situation is unpredictable and anything could happen.
For our last example, where better to find unpredictability than in the economy (2010):
A couple of scenarios occur to me. One is that the phrase would be used at the end of the period of taking bets: before spinning the roulette wheel (along the lines of "Les jeux sont faits", which I believe is the traditional phrase used there) or before the horses/dogs start. At that point, the bets are made, and yet anything can happen before the finish line.
The other possibility is that the projected outcome is so unpredictable that the house doesn't feel confident in taking any bet at all - it isn't even able to assign odds.
To me, the meaning has always been transparent: nobody would take any bet, since the whole situation is so uncertain.
I believe the complete phrase is "all bets are off the table" meaning that the event has so many variables that there is no way to have any form of organized gambling, and so spectators would make bets between each-other that cannot be declared on an organized gambling table. Similar to expressions like "nothing is off the table" and "getting paid under the table."
I'd say the connection is more direct, via another use of "bet".
"Tom is betting that his boss will be promoted next month."
This means that Tom firmly believes that his boss will be promoted next month and has taken action (or at least made plans) that can only succeed if his boss is promoted next month. Usually, this belief is the result of deep analysis, insight, or inside information.
It's not guaranteed, and implies that the one betting does not have direct control over events. Tom cannot promote his boss, and cannot control whether his boss will accept the promotion, but he is persuaded by his analysis, or rumors he's heard, that it is very likely to happen, and he's taking a risk based on that probability.
"The race has been cancelled, all bets are off."
What each bettor was persuaded would happen is now impossible. There is no race, and they no longer have a basis for their previous beliefs.
Thus, "The president has resigned, and now all bets are off", combines these: the race that was to be has now been cancelled, so the "bets" of those observing the race are now without meaning.