Each foray I've made into the Arctic and Antarctic has confirmed that low temperatures without the wind are relatively bearable, providing that your clothing is sufficient.
I'm writing this blog from Petermann Glacier in northwest Greenland, where a cold katabatic wind is blowing off the ice onto the deck of the Greenpeace ship Arctic Sunrise.