The phrase "it was all downhill from there" seems to have two, contradictory meanings.

The first indicates that things have since gotten a lot worse. For example (from http://bleacherreport.com/articles/2213152-jacksonville-jaguars-awards-at-quarter-mark-of-the-2014-nfl-season):

After a promising first half against the Philadelphia Eagles in Week 1 in which they jumped out to a 17-0 lead, it has been all downhill from there. They lost that game 34-17 and lost their next three games by a combined score of 118-41.

While the second means the opposite thing--that the hard part is over and it's all smooth sailing now. For example (from http://www.wesh.com/weather/hurricanes/sept-10-marks-peak-of-atlantic-hurricane-season/27985138):

September 10. That's the official peak of hurricane season for the Atlantic Basin. It's all downhill from here to the end of the season, November 30.

I pretty much exclusively use the phrase for the latter meaning, since to me that's more intuitive. If everything is "downhill" now, you don't have to pedal any more or work as hard to get to the bottom.

Can anyone elaborate on the origin of the phrase, what "downhill" is specifically referring to, the accepted usage, or if there are subtle differences in the way one is supposed to phrase the two different meanings?