It is just as plausible that selfish genery arrives rapidly at optimal designs, and that these shift only when what is optimal alters because, say, the environment has changed.
Gould, a palaeontologist, observed that there are long periods of stasis in the fossil record, which is true, and inferred from this that selfish genery is therefore wrong because it predicts continual change, which is questionable.