Robots.txt Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, whic
Download

Robots.txt Ranking & Summary

Robots.txt Tags

analyzer Crawler log analyzer create spider site editor Robots txt create TXT robots.txt generator robots.txt editor robots.txt manager edit robots.txt create robots.txt spider robots spider visits spider instructions visits

Robots.txt Description

Robots.txt is a visual editor tool for Robot Exclusion Files and a log manager package tool. Robots.txt allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, which parts of a Web site are not to be indexed and made searchable by the general Web public and then to identify spiders, which do not keep to those instructions. The program provides the user a way to log onto his FTP or local network server and then select the documents and directories which are not to be made searchable.

Robots.txt Related Software