As @LPH pointed out, be to and have got to (aka gotta) are indeed modal idioms, but they are unique conflations that include their own to complementizers as enclitics, whereas the other constructions in CGEL's list of Modal Idioms fall into a different semantic category. They're all involved with comparison, and what they're comparing is relative desires (of the speaker, usually).
All are followed by infinitives without to, which are licensed in every case (except had better) by the presence of an actual modal auxiliary in the construction.
- The first three are comparatives (note the -er suffix):
- would sooner
- had better
- would rather
- the next two are equatives (the as...as construction):
- would (just) as soon
- may/might (just) as well
- and the last one is a superlative (with the characteristic -st suffix):
Note that the equative phrases are all used to indicate a preference, not a lack of preference, between two verbs (normally, between doing something or not doing it, or between doing it one way instead of another).
As far as screwing around with the word order is concerned, idioms are tricky. By definition and in practice, they vary in how loose their constructions are. An idiom is like a TinkerToy wooden construction that's been left out in the rain. Mostly they won't come apart, except there's a slot where you can fit some kinds of things but not others. In had better/best, nothing comes apart.
- *You didn't have better/best done that
- *You hadn't better/best have done that
With modals and comparatives, you can get some things moved. I've heard both
as well as the much more common version
... but I'd rather not vouch for any more constructions.