[SAC] Google walking trac...

[Frank Warmerdam]

Frank Warmerdam wrote:
> Howard and I are putting robots.txt's into place that we home will screen
> direct access to svn and access to svn through trac.  Hopefully this will
> moderate our load issues.

Folks,

In follow up, we have put the following robots.txt files into place:

  http://trac.osgeo.org/robots.txt
  http://download.osgeo.org/robots.txt
  http://www.gdal.org/robots.txt
  http://buildbot.osgeo.org/robots.txt
  http://svn.osgeo.org/robots.txt

Hopefully this will reduce the search engine bot churn on our services,
while at the same time ensuring that appropriate content is focused on by
the bots.  Appropriate including wikis, drupal pages, and mailing lists.

When modifying and adding services, please keep robots.txt issues in mind.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org