In its article on "double act" (aka "comedy duo"), Wikipedia suggests that the "straight man" (or "feed," "dead wood," or "stooge") is paired with the funny man, banana man, or comic. In the Wikipedia article on "straight man," another possible name given for the counterpart of the straight man is comedian.
And yes, there is no reason not to refer to a woman playing the straight man/feed/dead wood/ stooge role a "straight woman"—although the only occasional straight woman I could think of right away was Anne Meara, who often played stolid affectless characters opposite Jerry Stiller's more manic characters in their comedy duo performances as Stiller & Meara. However, Meara and Stiller didn't play the same characters consistently, as, for example, Dan Rowan (straight man) and Dick Martin (comic) or Dickie Smothers (straight man) and Tommy Smothers (comic) did.
Frank Cullen, Vaudeville Old & New, volume 1 (2004) suggests that one of the earliest recorded straight women was Margaret Dumont, in numerous Marx Brothers movies:
Some comedy teams are quite strictly divided into comedian and straight man. Others are not. Relatively rational, compared to the comedian, straight men are the stand-ins for the audience. They anchor the routine in plausibility, which the comedian constantly confounds. The job of the straight man is not to be funny but to ensure that the comedian gets laughs. In the most direct case, the straight man functions as an interviewer, as much in control of the questioning as they are of the pace of the act. ...
Playing straight may also be the role of a comedian's foil. Margaret Dumont is a good example. Most notable was her work with Groucho Marx. Although Dumont was supposed to appear utterly confounded and distressed by Groucho, her delivery, timing and ability to hold for laughs defined her as a straight man (or straight woman) and a feed.
A quotation attributed to Margaret Dumont at the beginning of this YouTube video of film clips of Margaret and Groucho says,
I'm not a stooge. I'm the best straight woman in Hollywood. There's an art to playing it straight. You must build up you man, but never top him, never steal the laughs from him.