I think 'drawn' means 'showing the wearing effects of overwork, care or suffering', but I can't understand what 'you are drawn to her' will be meaning right here.
In this context, "to be drawn to a person" means "to be attracted to a person". This does not necessarily mean sexually or romantically, but it is often used that way.
You may be thinking of "drawn" in the artistic sense, as in making a sketch, but that would be phrased as "you are drawn by her". It is also possible to use "drawn" in the sense of "attracted" with "by", as in the sentence, "He was drawn by her passionate denunciation of the political corruption."
EDIT - Per Catija's comment, if the adjectival form of drawn was intended, it would probably be in the form of "You looked (or appeared) drawn to her." The past tense would be used, since the speaker would presumably be passing on the results of a conversation with "her", since there would be no other way to know how she interpreted your appearance.
to be drawn is simply the passive form of to draw meaning to pull or to attract.
Active: John drew the phone towards him.
Passive: The phone was drawn towards John (by John).
Active: John was so good-looking that he drew all the women towards him as if by magnetism. (John attracted the women)
Passive: John was so good-looking that all the women were drawn towards him as if by magnetism. (the women were attracted by John)
If you are drawn to someone, it means they have a quality that draws (pulls/attracts) you to them.