I happened to meet her this morning.
I've learned 'happen to' means to do something by chance, so the sentence means I met her this morning by chance.
A: I never seem to do anything right around here. B: I happen to think you do a lot of things right.
It's one of the lines in a movie, and someone told me 'happen to' here in the movie means "actually." I've looked up a lot of dictionaries, but I've never seen the meaning of "actually" for 'happen to.' What does it mean in that sentence, and if it means "actually," then is it used that way commonly?