I believe a fairly standard label for this category that is more precise than "lexical quantifiers" would be "nonnumerical quantifiers"; for words that express exact or approximate numbers but are not themselves words for numbers or related to words for numbers.
Not surprisingly "nonnumerical" means that the terms in this category are not derived from any other words that convey a number or numerical quantity at all. I gleaned this definition from this article.
By this definition, "score" is a nonnumerical quantifier, as is "gross", but I would have to take issue with "dozen".
By the same definition, "dozen" probably IS a "numerical quantifier" and not "nonnumerical" in that "dozen" is derived from the same roots as the Latin-origin equivalent of our word for "twelve" (which is Germanic). In Latin-origin languages the similarity is plain, e.g. "douze" (French) "doce" (Spanish) and "dodoci" (Italian). So I would put "dozen" sort of in between numerical (e.g. "trio" related to "three") and nonnumerical (e.g. "brace" for "two") quantifiers, depending on whether the quality of being numerical is to compare it to words for numbers in only the same language as the term in question.