In answer to the question, "Do we need any more chairs?" why does it sound correct to say, "No, three's enough." Why does "three are enough" sound wrong?
This is a Use-Mention Problem, and we compound it by talking about it here.
And even more so by writing about it and not talking.
So we'll have to be very careful about the quotation marks, OK? I'll take it slow.
First, three is a word, and it is a plural word, since it denotes a number larger than one. Hence,
Second, three is a number, and numbers have names.
Third, the name for the number three is "three". With quotes.
Fourth, the name "three" is also a word in English (pronounced just like the number three).
Fifth, since it is the name of one thing, the name "three" is a singular word. Hence,
This last sentence essentially means
Therefore, in many situations we have the possibility of referring to any number either as a single unit quantity (by referring to the name of the number), or as a plural set (by referring to the number itself).
Philosophers talk about this as a distinction between Using a word (that's the plural number three), and Mentioning it (that's the singular name "three"), and they can get quite testy about confusions involving them.
Three is singular. There is a single set of three, and that is enough. Or rather, the single number three is sufficient.