It's not particularly common for expressions of time.
It's similar to degrees-minutes-seconds: instead of decimal degrees (38.897212°,-77.036519°) you write (38° 53′ 49.9632″, -77° 2′ 11.4678″). Both are derived from a sexagesimal counting system such as that devised in Ancient Babylon: the single prime represents the first sexagesimal division and the second the next, and so on. 17th-century astronomers used a third division of 1/60th of a second.
The advantage of using minute and second symbols for time is that it obviously expresses a duration rather than a time.
From the time 01:00:00 to the time 02:34:56 is a duration of 1 hour, 34 minutes and 56 seconds (1h 34′ 56″)
Prime markers start single and are multiplied for susbsequent appearances, so minutes use a single prime ′ and seconds use a double-prime ″. They are pronounced minutes and seconds respectively in the case of durations like this.
Note that a prime ′ is not a straight-apostrophe ' or a printer's apostrophe ’, although straight-apostrophes are a reasonable approximation and printer's apostrophes do occur as well.