The structure I have known someone to do something is apparently considered grammatical and idiomatic. Examples from Google:
I have known people to take shops, put in a few articles and, without opening the doors, to sell the premises for as much as £1,000. (New Zealand Parliament)
I have known people to temporarily lose interest in sex and believe the problem was in their marriage but later realize that it was grief.
I have known people to lose battles with brain tumours, cancer, to become disabled, and to become very ill.
In contrast, I know/knew someone to do something is not a thing that people say. Apparently you can't say "I knew some people to drink coffee around midnight.*" Why is that? What is special about the structure I have known someone to do something that validates it as grammatical?