Depending on whether an object hitting us were a fast-moving comet or a slower asteroid, it would need to be about 2.5 miles across to set in motion the process of reducing light and destroying photosynthesis, without both of which Homo sapiens and the other forms of life on our planet could not survive.
FORBES: Space: Our Ticket to Survival