# robots.txt for arXiv mirror sites http://*.arxiv.org/ # Robots should not harvest from arXiv mirror sites, see http://arxiv.org/RobotsBeware.html # $Id: robots.txt.mirror,v 1.5 2012/04/27 15:59:27 arxiv Exp $ User-agent: * Disallow: /cgi-bin/ Disallow: /e-print/ Disallow: /src/ Disallow: /ps/ Disallow: /psfigs/ Disallow: /dvi/ Disallow: /year/ Disallow: /pdf/ Disallow: /html/ Disallow: /cookies/ Disallow: /form/ Disallow: /xxxform.html Disallow: /abs/ Disallow: /find/ Disallow: /view/ Disallow: /ftp/ Disallow: /refs/ Disallow: /cits/ Disallow: /list/ Disallow: /format/ Disallow: /archive/ Disallow: /register Disallow: /submit Disallow: /replace Disallow: /cross Disallow: /jref Disallow: /e-find/ Disallow: /paper_passwd/ Disallow: /PS_cache/ Disallow: /Stats/ Disallow: /cmp-lg/ Disallow: /seek-and-destroy Disallow: /IgnoreMe Disallow: /uploads Disallow: /auth Disallow: /catchup Disallow: /tb Disallow: /tb-recent Disallow: /trackback