My Phone charges when it's off at first then shows a caution sign

Okay so my phone has recently become more difficult to charge. (For example it will only ever really charge when turned off.) I figured I could live with it. But for the past few days when I set my phone down on the charger for the night it will charge at first but when I wake up I find that the charging symbol shows a yellow caution sign. It charges it to around 60 then stops. And the first time it did this I restarted it and it jumped from 60 to 85. I figure the battery may be going bad but I would like a second opinion.