This is obviously a subjective issue. The two existing answers both endorse OP's usage, but I don't.
There are various contexts where you can reasonably use a figurative temporal reference in a superficially "scalar" sense - for example, the early pages of a book (though for reasons I can't pin down, the late pages doesn't work for me, only later). But that's because we normally start reading a book at the beginning; page numbers increase as time passes.
For contexts like OP's "number of shoes" I can only envisage using a temporal reference if there's a similarly obvious link between the passage of time and an increasing scalar value. For example,...
"How many blu-ray movies do I have? Let's see - I got my blu-ray player about three years ago, and I add another disc to my movie collection every month or so. It must be up to the late thirties by now."