I googled and looked up for sentence examples and similar expressions but found nothing, so this is the last place I'm asking this. Does "In face of all that" work as an alternative to "In conclusion"? Could anyone suggest other ways to say that?
"in face of all that" should be more correctly written as "In the face of all that", which means "In opposition to all that" or "despite all that", or "presenting a different opinion to all that".
Obviously people would expect "all that" to have been defined, usually in the preceding sentence or paragraph. For this reason, it's not a good way to start a whole new section of a text: there's too much of a break since the previous section. The reader would be left wondering what "all that" referred to: the last thing mentioned in the previous section? The entire previous section? The entire text up to that point?
Why don't you want to simply say "In conclusion"? That would seem a better choice.