When I say, read it or drink it or take me, there is no to in-between. Why is it that when I use the verb listen, I have to say listen to me or listen to it?
The verb "listen" takes a prepositional object, while a verb like "read" takes an 'ordinary' direct object.
Why should this be so? Well, to some extent, "because it is": languages allow prepositional objects-- and English allows them readily-- and so some verbs fall into that paradigm. The mechanism for them arising is probably that originally, the preposition has more of its "full" lexical value (e.g. maybe when a person listened "to" another, there was originally the idea of them literally turning their ears towards them), but over time becomes grammaticalised-- i.e. in effect, people become so used to a word occurring in a particular situation, that they become desensitised to its meaning and just "expect" it to be there in that construction. It's in effect how the going-to future arose in English: once upon a time "I'm going to get some water" would have literally implied "I'm going to another place where there is water", whereas now it could mean "There is some water right here where I am and I will now take some of it".
The arbitrary nature of prepositional objects can be seen in the fact that different languages may have them or not to express a particular concept (English "wait for", but in French "attendre (après)", the preposition "après" is optional-- and indeed stigmatised in careful usage), and that within the same language, different verbs expressing the same notion need not share the same argument structure (so "wait for...", prepositional object, but "await ...", prepositionless).
Incidentally, "listen" and other verbs with a prepositional object are still arguably transitive.
You should listen up; I don't want to have to say this twice.
Whether you listen from over there or whether you listen while laying down, you will hear what I have to say and you will understand eventually. Just because you may listen, does not mean that I must listen as well.
Do you see what I did there?