I'm looking for a word that would be roughly equivalent to "the process of becoming a taboo," specifically in the context of film/literary criticism, where the "taboomanization" of sexuality serves as a common literary theme in the type of fiction I'm writing about.
tabooization is used:
Here is a random example from The International Politics of Sport in the Twentieth Century By James Riordan, Arnd Krüger:
Along with it arose a strong 'tabooization' of sexual themes generally known as Victorianism.
Probably "demonization" is the closest we've got. When Captain Cook brought the word "tabu" back from Polynesia, it caught on quickly in English because it's a darned useful term. Another term is "Bowdlerization," which refers to the act of deleting obscene or forbidden material in text, and often has literary applications.