You might say "one woman man", meaning for that particular man, there was only one woman. Usage: He was a one woman man, and after his wife died, he never remarried.
Equally you could say, "one man woman", meaning for that particular woman, there was only one man. Usage: She was a one man woman who remained single after her love married another.
In "one woman man", the phrase "one woman" acts as an adjective to describe the man.
In English I am unaware of a single word to convey the meaning you require, other than monogamous.
Additionally, in English, monogamous is not gender specific. A woman can be equally as monogamous as a man. Usage: Both Bill and Mary were monogamous. Luckily they were married to each other.