In non-computerese it doesn't need to refer to a single unit, though often it does.
Etymonline says it comes from
mid-15c., "act or process of increasing," from Latin incrementum "growth, increase; an addition," from stem of increscere "to grow in or upon" (see increase).
We often refer to something increasing by a small amount as "increasing only incrementally" to stem the notion that the increase was great or significant. But the increase doesn't have to be a single unit, and most often it is not. Sometimes a single unit doesn't even make much sense, as when a probability increases, or something increases by a percentage.
In programming, however, when you increment a variable it implies 1 unless there is a specific other amount. (Cf. the "increment" operator in C-syntax languages. The
++ always means
+=1 and the decrement operator (
--) always means
-=1.) When might there be such another amount? When you need to quantize values so that, say, a 3 or a 6 must round up or down to produce always a "five" block. This is by far the exception, however. Absent any other information, you should assume you must increment by 1. Note also that when there is a specific other value to be incremented, that amount will, or should be, explicit: e.g., "increment by 5."