Take the new battery and orient it correctly -- the arrow should be facing towards the camera lens, face up.

Insert the battery at an angle, pushing the battery in until it is aligned on the top edge. Push the rest of the battery in, snapping it into place.

Reattach the battery cover, and power up your phone.