Remove the bumpers by prying them off of the pegs that secure them, using a spudger. They are located on the front and back of the controller.

A standard nylon spudger is recommended for this step. Using a metal spudger near the motherboard as shown risks damaging your device.