[SAC] Re: [OSGeo] #277: Robots Are Attacking!

[OSGeo]

#277: Robots Are Attacking!
------------------------+---------------------------------------------------
  Reporter:  warmerdam  |       Owner:  sac at lists.osgeo.org
      Type:  task       |      Status:  new                
  Priority:  normal     |   Component:  SAC                
Resolution:             |    Keywords:  trac               
------------------------+---------------------------------------------------
Comment (by jbirch):

 I wonder if it would be worth setting crawl-delay for the major spiders?

 Yahoo and Microsoft support this directive in robots.txt, while for Google
 you have to set up a Webmasters Tools account and tell it to slow down in
 there.

-- 
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/277#comment:2>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.