I'm going to go against the grain of the current answers, and say that this is indeed wrong. That said, I should qualify that in a couple of ways: first, whether something is grammatically correct is dependent on a number of factors - what is acceptable in colloquial speech may not be in formal writing, and what is acceptable in one region may not be in another; what's more, there is no "language police", and accomplished writers often "break the rules", either to achieve a particular effect, or because doing so allows them to express their meaning more clearly - and there is nothing wrong with this. It's also worth noting that the original context is a lecture, and therefore speech (albeit more formal than conversation) rather than formal writing, so criticising it in the way I am about to is somewhat unfair.
With that out of the way, I would advise against this in formal written contexts (though it is absolutely fine in speech, where sentence fragments and disjointed sentences are the rule rather than the exception!). My reasoning is as follows:
The word "which", in this context, is a relative pronoun (functioning as a determiner). As such, it functions as the link between something referred to in the main clause and the same item in a subordinate clause:
I am learning mathematics, the study of which subject goes back to ancient times.
Here the relative pronoun is used to join together the two sentences
- I am learning mathematics.
- The study of this subject [i.e. mathematics] goes back to ancient times.
with the first as the main clause, and the second as the subordinate.
To look at the example in the question:
You may find the moral category too severe [...]. In which case, you may want to change the title [...].
Here, the second sentence contains only a subordinate clause, with no main clause - and this is what is "against the rules", so to speak - and many readers may find it jarring. It is parallel to:
*That is a dog. Which I saw yesterday.
...which is much more clearly wrong.
Joining the two sentences with a comma (or perhaps a dash to provide greater separation, since in this case there are already several clauses separated by commas) remedies this.
[ On a separate note (for clarification): this usage of "which" - as a determiner qualifying a noun - is different from and much less common than using it as a pronoun in its own right, for example:
I am learning mathematics, which is a difficult subject.
In both cases the "which"-clause is subordinate. ]