On 31-Jan-07, at 1:24 PM, Jason Birch wrote: > Perhaps an "allow from" entry for that file would be better, set to > only > allow from the machine that triggers the cron job? > > robots.txt is only advisory Makes sense to me, but out of my realm. Someone else?