Right ear side, line removal

Having one working/hearing ear, on the lovely Left-hand/ear side of my iPhone headphones, the opposite side of the remote (BOOOOOO), I would like to remove the right-hand/ear side, as aside from the remote that I continuously use, it only gets in the way. Can one side of the headphones be removed, or will doing that cancel the sound, and can the parts of the headphones that sit in the ears be swapped, one side with the other, which would enable me to use the remote if/when the opposite line is removed?

THANK YOU!!!