Is it correct to say this?
Her not paying attention to the class annoys me.
Yes, it is correct to have such a possessive structure as a subject:
The Other Side of the Couch: The Healing Bond in Psychiatry Gail Albert - 2011
Meanwhile she's already called back twice. Her not leaving a last name suggests that she believes I know her when I don't. And the number of calls suggests that she feels I'm a very important—and powerful—figure in her life.