You left off on page 50 (of the book I'm teaching).
This sentence is grammatically correct in every way.
To leave off is a phrasal verb meaning "come to an end, stop or cease" (Princeton WordNet)
The correct preposition "on" is used:
ON (meaning #4): used to say where something is written or shown:
There’s a diagram on page 25.
(Longman)
- There's no definite article before page as it should be omitted in this case:
There is no article before a noun followed by a categorizing letter or
number:
The students have just read section C.
The Chicago train is about to depart from track 5.
Her flight leaves from gate 32.
He fell asleep on page 816 of "War and Peace".
She is staying in room 689.
(http://www.davidappleyard.com/english/articles.htm)