<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><DIV><DIV>On 29-Mar-07, at 8:45 AM, Jason Birch wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"> <DIV dir="ltr" align="left"><SPAN class="311002415-29032007"><FONT face="Arial" color="#0000ff" size="2">Not sure that I get a vote, but I reluctantly have to agree with Howard. Is there a way that we can allow the bots to at least spider the main page, but ignore anything else?</FONT></SPAN></DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV>Isn't there a way to feed search engines our site map directly?</DIV><DIV><BR><BLOCKQUOTE type="cite"> <DIV dir="ltr" align="left"><SPAN class="311002415-29032007"><FONT face="Arial" color="#0000ff" size="2">But what are the options? As Howard has pointed out, we have limited volunteer server admin resource, so what can we do with what we've got? </FONT></SPAN></DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV>I'm also trying to find more people who are familiar with tuning drupal. (Swapnil keep the ideas coming and put into the wiki!) Ejer and alienbrain (on IRC) may be able to help too.</DIV><DIV><BR><BLOCKQUOTE type="cite"> <DIV dir="ltr" align="left"><SPAN class="311002415-29032007"><FONT face="Arial" color="#0000ff" size="2">Another thing that should be looked into is why the server is responding in a way that invalidates caches. This makes the bots see our content as highly dynamic, and the'll come back for it way more frequently.</FONT></SPAN></DIV> </BLOCKQUOTE></DIV><BR><DIV>Cache was currently set to 5min lifetime, perhaps it should be more now.</DIV></BODY></HTML>