It was the perfect barrier to entry.

FORBES: Forget Apple's iPad, Publishers Need To Worry About Tigers

All the search engines follow these robot directives, which, while not a perfect security solution, limit the entry points used by the bots and spiders that index the Web.

FORBES: Google Me Not