(There are others, such as table, paste, and baste.) The rule I've heard is that a vowel is made long when succeeded by a consonant and then another vowel. Some words treat double consonants as a single consonant for this rule - hence able has a long "a".
Why is this, and is there a rule governing such words?
In the case of haste, the general rule is that an E at the end of the word makes the preceding A (before 1 or 2 consonants) long. Paste has a long A, but past has a short A. Baste and waste have long A's, but fast, last, mast, and vast have short A's. Unfortunately (as often happens in the English language), there are occasional exceptions: caste has a short A.
The first rule of English is that there are very few rules that are always kept.
English is a mongrel language with conflicting conventions of pronunciation. The best one can do is attempt to be consistent. And acknowledging that as a living language English will always be inconsistent. Quel dommage.
Historically, the pronunciations of most words are not derived from their spellings: "able" and "haste" are pronounced with the "long a" sound because of the way they developed, not because of the way they're spelled. In English there isn’t even a reasonably simple correlation between spelling and pronunciation, as there is for some languages. Nonetheless, there are relevant spelling patterns here.
Certain vowels and certain consonant clusters behave differently with respect to the “long vowel before silent e” rule. In particular, sequences of a single consonant followed “l” or “r” often do not shorten the preceding vowel.
Some examples:
-bre in fibre, sabre (mainly British English)
-tre in centre, metre, nitre, mitre (mainly British English)
-cre in acre
-gre in ogre
-ple in maple, staple
-ble in stable, able, Bible, noble
-tle in title
-dle in cradle, bridle, idle
-cle in cycle
-gle in ogle, bugle
Note that for many of these, there is a corresponding ending with a doubled consonant that does indicate a preceding short vowel (-pple as in apple, -bble as in dribble, -ttle as in battle, -ddle as in saddle, -ckle as in buckle, -ggle as in struggle).
And obviously, a sequence like -mple or -ngle will also tend to be preceded by a “short” vowel.
There are a fair number of exceptions to this pattern, such as couple, trouble, double, treble, triple.
The situation with “aste” is a bit different. Vowel length in words ending with the sound “st” is a bit variable, but the variation in spelling depends a lot on the specific vowel sound.
Words with a “long e” sound before “st” are generally spelled with “east”, using one of the usual digraphs for the “long e” sound: "ea".
Some words with a “long o” sound before “st” are spelled with “oast”, using the usual “oa” digraph for the “long o” sound, but many don’t get any special marking. So we have “roast” and “coast,” but also “post, host, most” with nothing to show that the spelling is different from “lost, cost, frost.”
Words with a “long u” sound before “st” don’t really exist, as far as I know. There are words with “oo” before “st”, which are simply spelled with “oost”.
Words with “long i” before “st” are rare and spelled various ways, since there is no usual digraph for the “long i” sound. The spelling “eist” probably occurs in the greatest number of distinct words, but none of them are very frequent. The ambiguous “ist” occurs only in the common word “Christ”, while “yst” occurs only in the word “tryst," which may also be pronounced with a short vowel.
So in fact, “a” is the only vowel where it is regular to indicate a “long” pronunciation before “st” with a final silent “e” (probably this method is used because our standard English spelling system has no usual digraph for the “long a” sound; "ai" as in "waist" would not have been appropriate in Early Modern English as it had likely not yet merged for many speakers with the "long a" vowel). We find this -aste pattern, as other answers have mentioned, in words such as paste, baste, waste, haste, chaste. The word caste, pronounced with a short vowel, was spelled “cast” before 1800 according to the Oxford English Dictionary; the “e” doesn’t indicate a long vowel here, but is simply an alteration of the word to the French spelling.
We see “ste” after vowels other than "a" in some other French loanwords, especially those that are recent or that are still not entirely assimilated, such as “artiste,” “piste”, “celeste”; in this context it doesn't tell you anything about the pronunciation of the vowel, except for suggesting that it will be stressed and probably approximately what English speakers think the French vowel sounds like. (In fact, a great deal of the words I mentioned previously are also ultimately from French, but words like “taste” and “roast” have been assimilated into English to a greater degree than words like “artiste”).
