All the search engines follow these robot directives, which, while not a perfect security solution, limit the entry points used by the bots and spiders that index the Web.
The real question is to what extent branded companies such as Amgen which are also developing biosimilars want to limit the future role of these products or simply raise the barriers to entry.