You are not entirely wrong. According to this Wikipedia article 'antonym' is
The term antonym (and the related antonymy) is commonly taken to be synonymous with opposite, but antonym also has other more restricted meanings. Graded (or gradable) antonyms are word pairs whose meanings are opposite and which lie on a continuous spectrum (hot, cold). Complementary antonyms are word pairs whose meanings are opposite but whose meanings do not lie on a continuous spectrum (push, pull). Relational antonyms are word pairs where opposite makes sense only in the context of the relationship between the two meanings (teacher, pupil). These more restricted meanings may not apply in all scholarly contexts, with Lyons (1968, 1977) defining antonym to mean gradable antonyms, and Crystal (2003) warns that antonymy and antonym should be regarded with care.
So, in order to answer your question. You can call such binary pairs of terms opposite words.
From Cambridge Dictinary
something or someone that is completely different from another person or thing:
e.g. The opposite of "fast" is "slow".
Again, from same wiki article
In lexical semantics, opposites are words lying in an inherently incompatible binary relationship, like the opposite pairs big : small, long : short, and precede : follow. The notion of incompatibility here refers to the fact that one word in an opposite pair entails that it is not the other pair member. For example, something that is long entails that it is not short. It is referred to as a 'binary' relationship because there are two members in a set of opposites. The relationship between opposites is known as opposition. A member of a pair of opposites can generally be determined by the question What is the opposite of X ?