The CMOS battery has to be removed properly to avoid damaging the motherboard.

Using either your fingers or a spudger, press the small metal tab securing the battery to remove it.

Insert a new battery, reattach the side panel, and you've successfully replaced the BIOS battery.