The affinity of the negation is to the verb not the the 'much'.
You can tell this is going on because you are more likely to see "That isn't much use to me" than to see "That is little use to me." And when you answer 'Not much', you are paraphrasing 'Yes, there is. But there is not much'. If there were none, you could not say 'not much', though none is surely within the opposite of 'much'.
Negation in English has odd binding qualities. For instance modal verbs like 'might' have two opposites: 'You might not' means it is possible you won't, but the real opposite of 'you might' would be that 'you must not', the impossibility that you will. For that reason, by convention, 'not' binds to the verb in a way that favors the more common negation. (Many a child has played this game with 'you may not', artificially mistaking it to imply that it is allowed that you might not, and not that it is forbidden that you might.)
'Is' borrows this disease. The common negation denies the truth of what is predicated rather than asserting its opposite. 'There is not much' does not necessarily imply 'There is little', as it would if the 'not' really went with the 'much'. But it allows for amounts that are middling, because it is only denying 'There is much' not asserting there is the opposite of 'much'.
Games and silly idioms arise as a result, including the feeling that 'not much' is a quantity in itself, greater than 'little', but less than 'much'.