Penultimate is a very mannered way of saying second to last, and its use in English goes in and out of fashion. According to Google ngrams, it has been rarely used in American English since around 1840, and in British English it marks certain periods, as shown below.
Personally, I think of it as Victorian (1837-1901), and you can see a marked peak in 1865, but it seems to have lived on. It seems to be passing out of fashion again, although it's too early to tell.
It's a useful word if you're hoping to establish a pedantic tone, e.g. in an academic paper that needs additional weight and dryness. Otherwise not so much.