It used to be that "master" was the word for a man who was in authority or in control, and "mistress" was the word for a woman in such a position. I presume that "mistress" came to be used for a woman that a married man was having an affair with on the idea that she is controlling and ruling him through her seductive powers. This usage has come to overshadow the literal meaning of the word, so that today if you use the word "mistress" in the old sense people will almost inevitably think of the sexual connotation. Like, today if you say, "Fred is the master of the soccer team", people would understand you to mean that he is a coach or star player or whatever who exercises a high degree of control. But if you say, "Sally is the mistress of the soccer team", people would think you meant that she was having an affair with every man on the team.
I once read a post on a web site in which the woman who ran it said she didn't like being called by the masculine term "web master", but when she called herself the "web mistress" the guys in her organization had way too much fun with the term.
In common use, the male equivalent of "mistress" is "lover" or "boyfriend". Those aren't exact equivalents, though. "Mistress" is usually used for the unmarried girlfriend of a married man who is supporting her financially. "Lover" could apply to either sex with no implication whether either is married to someone else. "Boyfriend" indicates a romantic relationship that may or may not involve illicit sex, again without any implication of the marital status of either party.
I can't help but add: I once heard a comedian comment that he stumbled across the word "nymphomaniac" in a dictionary, and it defined it as "a female who is completely obsessed with sex". So, he said, he wondered if there was an equivalent term for a male who is completely obsessed with sex. He did a little research and he found that there is. The word is "man".