Someone who has been worn down by a harsh working-life?
I think haggard fits the bill here.
having a gaunt, wasted, or exhausted appearance, as from prolonged suffering, exertion, or anxiety; worn
I think toiler may suggest the idea of a hard worker:
Toilers are people who work long and hard. Any employer would be grateful to have toilers as employees.
If you know that to toil is to work hard, then the meaning of toiler won't be a surprise. A toiler works strenuously; you won't see a toiler hanging out by the water cooler. People with physical jobs such as construction workers and miners are often referred to as toilers.
someone who works diligently for long hours. (TFD)
Well, yes, exactly:
abused or oppressed by people in power
... but you'd be using it in a more literal and less figurative sense. Worn down by work with little reward.