format robots.txt

[Pavel Urban]

viz http://info.webcrawler.com/mak/projects/robots/faq.html#robotstxt a
okoli

Michal TALMAN wrote:
>
> Co musi obsahovat soubor robots.txt, ktery vyhledavaji agenti vyhledavacich serveru, aby mohly byt tyto stranky uspesne zarazeny do databaze onoho vyhledavaciho serveru ?
>
> Diky
>
> M.T.

***********************************************************************
Pavel Urban (urbanp at mlp.cz)
MLP system disaster
Mestska knihovna v Praze - Marianske nam. 1, 115 72 Praha 1, Ceska
republika, http://www.mlp.cz
***********************************************************************
   Vegetables should not operate electronic equipment.
          Computer Stupidities, http://rinkworks.com/stupid/
***********************************************************************