LangaugeLanguage has very little to do with logic.
If the meaning is clear - and I think it is, here - what else matters?
I don't think I'd say ten times fewer, but I understand what people mean by it, and (I think) why they say it that way: they're used to expressing quantities by multiples ten times bigger, ten times as quickly. Would you have the same objection to ten times slower?