If the old battery is still in the slot, then make sure the camera is turned off prior to removal.

If there is a battery currently in the slot, remove by first pulling downwards on the blue component. This will allow you to remove the battery from the slot.