The L1 and R1 buttons are held in by two clips. If only one clip releases, move your pick to the other side of the button to release the remaining clip. Don't continue prying on one side after a clip is released, or you risk breaking the other clip.

Use the pick to pop both the R1 and L1 shoulder buttons out.

Use your thumb to prevent the button from flying across the room.