They are grammatically correct. It's a sort of interjectory phrase, intended to call attention to the particular intensity of a description. This construction is often used for poignancy in poetic language, as well as in compliments to younger individuals ("How smart you are!", "How tall he's gotten!"), but I'm not sure that there's actually a name for it. In conversation, a person may say something like "How interesting..." as a prompt for you to go on—though it may often be heard in a sarcastic tone for its opposite meaning. That usage is short for "How interesting that is!".
Your examples are akin to "It was so interesting to see him go!" and "I felt so dreadful when I heard those words", but with slightly more emphasis on the sentiment than on the subject.