Well, it depends on the man. And not just in the obvious way that we all find some people more pleasing to the eye than with others, but in that, with both men and women, it suggests a certain striking handsomeness that stands out, beyond merely good-looking.
If I say "Tom Hiddleston* is a looker", I don't mean "I personally find Tom Hiddleston attractive", but more "Tom Hiddleston is so obviously handsome that even if he doesn't 'do it for you', you'll see why he does for other people".
It is a term that was originally applied much more often of women rather than men, which comes from a history in which women's physical attractiveness was commented on much more than men's.
It's now less often seen as appropriate to talk about women's attractiveness in contexts where it would once have been accepted. Conversely in those contexts where such comments are acceptable, it's now both more acceptable for women to state what they find attractive as vocally as men, and more acceptable for men to state that they find other men attractive.
The bias is still toward using it of women rather than men, in part because there remains a bias toward the degree to which physical beauty is valued in women rather than men, and in part because of qualms of some straight men in discussing men in this regard.
Hence while straight, gay and bisexual men and women might all say "Maggie Gyllenhaal is a looker" though with different types of responses to the attractiveness described, it would mostly only be women and gay or bisexual men who would say "Tom Hiddleston is a looker" with straight men more likely to say something less emphatic, if they said anything at all.
Of course, there are exceptions to that.
It is a bit of a dated term too, so mostly entirely other term are likely to be used
*Examples are for illustrative purposes, your opinion of Maggie Gyllenhaal and Tom Hiddleston may vary.