# robots.txt file for www.macs.hw.ac.uk # this files tells automated search robots which directories # they shouldn't try to index (if they're well behaved) # This is aimed at all search agents: User-agent: * Disallow: /cgi-bin/ Disallow: /elasticsearch/ #