Keeping a second home is a step up in magnitude because a second home has all the costs (often more) of your first home without the easy write-offs from the IRS. By and large, second homes are often a terrible financial burden rather than a good investment.
After a hurricane, for example, large trees that are snapped or sheared often become pulp wood rather than saw-timber, reducing their value by two-thirds or more.