The expression probably comes from the physical act of turning your back toward someone, instead of speaking to him face-to-face.

VOA: special.2009.12.13

If someone is detained because they appear ill, a medical expert from the Centers for Disease Control and Prevention comes to the scene to check them and, if necessary, send them back over the border.

VOA: standard.2009.04.30

So if you're asking me I want you to know When my time comes Forget the wrong that I've done Help me leave behind some Reasons to be missed Don't resent me And when you're feeling empty Keep me in your memory Leave out all the rest Leave out all the rest Forgetting all the hurt inside You've learned to hide so well Pretending someone else can come And save me from myself I can't be who you are When my time comes Forget the wrong that I've done Help me leave behind some Reasons to be missed Don't resent me And when you're feeling empty Keep me in your memory Leave out all the rest Leave out all the rest Forgetting all the hurt inside You've learned to hide so well Pretending someone else can come And save me from myself I can't be who you are I can't be who you are (SINGING) How can I decide what's right When you're clouding up my mind I can't win your losing fight All the time How can I ever own what's mine When you're always taking sides But you won't take away my pride No, not this time No, not this time How did we get here?

VOA: standard.other