Almost all of the at-large spots are assigned to also-ran teams in the big-money conferences who control the NCAA. Even teams that finish as low as eighth in a conference are invited in.
It may be about to happen in particle physics - if neither the Tevatron nor Large Hadron Collider spots a Higgs boson soon, the standard model that scientists have worked with for years may have to be abandoned, or at least seriously reformed, and another built in its place.