This shows what your device looks like on the inside behind the part that you just removed from the headset.

In the red square is the connection between the battery and the headset.

Gently pull the wires that are connected to the white part in the red square.

Now that your battery is not connected to anything you can take it out of the headset.