I like these phrases that can be viewed in any way you like. Downhill from here can mean things get easier; but it can also mean things get worse. 
Pete is 60 today - Happy Birthday - take it whichever way you like - But I would always take the positive option if I were you..