Assuming that you meant "matter of fact" (as opposed to "matter of the fact"), there is a difference between the two idioms.
"Matter of fact" (often used in a sentence as 'as a matter of fact, ...") is described by Mirriam-Webster as:
adhering to the unembellished facts; also : being plain, straightforward, or unemotional
Often the phrase is used to explain that a previously stated 'fact' or detail which was thought to be unlikely actually is in fact true or alternatively to counter a previous statement. Examples (from The Free Dictionary:
"As a matter of fact, John came into the room while you were talking about him."
"I'm not a poor worker. As a matter of fact, I'm very efficient."
"Fact of the matter" means simply "the truth". It's typically used to emphasize a fact (perhaps one that has been debated) but which isn't really a valid thing to debate about.
"The fact of the matter is that unless we drive faster, we wont arrive on time."