Yes, it is an investment of time and effort to have lunch (or breakfast or afternoon tea) with your direct reports and get to know them as people.

FORBES: Are You Ready For Your Most Valued Employees To Leave?