It appears that adversity in childhood is associated with facial features that are not perfectly aligned and matching, although there's no proof that one of these phenomena causes the other.

CNN: Beholding beauty: How it's been studied