Littlest is not 'standard', but perfectly acceptable in conversation. What you should be concerned about is that it has overtones of cuteness and childish language - consider The Littlest Mermaid, or Google the word and see the contexts in which it is used.
I think you will usually want to avoid it. However, the same can be said of its 'standard' alternative, least, which is strongly marked for formality. Except in fixed phrases like the least of your concerns or I haven't the least idea, where least is obligatory, practically nobody uses least colloquially to mean the superlative of little (or less for its comparative).
Smallest is safest. Smallest will never make you sound like either a kindergarten teacher or a college English professor emeritus.