Ideally you want to keep the stereo image as intended. Once you start changing that theme you lose all reference points all the way down the chain when troubleshooting and that probably is the biggest drawback. Now going back to old songs the you know, would also bother you while listening knowing what side of the track things should be on!

As far as L-R, Every mix is different as far as instrument placement and that's clearly up to the producer or mix engineer. When I recorded and mixed records for a over a decade it was all about how we set up the musicians in the live room or how I decided I wanted to place things in the stereo space to get the best possible clarity and "space" in the mix. I would say wiring things out of phase would have a more dramatic and negative impact then L-R balance...as far as sonics go(although we often had certain channels in the mix out of phase to create space as well).