external searching bugzilla database, robots.txt

[Gervase Markham]

Jason Remillard wrote:
> Now that I have a clue, I see that there is a bug (81920) against not 
> letting in search engines. It is marked as “will not fix”. I am not sure 
> if it was marked as do not fix because letting in search engines was 
> considered a bad idea, because the solution suggested in the bug was 
> bad, 

It's a bad idea - because the search engine will display out-of-date 
information, and because having search engines crawling the site would 
probably be a significant load that we can't bear.

> or because it was not going to be done for the current release (2.16).

If that were true, its target milestone would just have been changed.

> Obviously I think that it is a good thing to unlock the wealth of 
> information the bugzilla database to the search engines. Also, many 
> companies have internal search spiders, so I think the value in indexing 
> holds up for both public and private installations. I am sure it would 
> have saved me time if the public bugzilla database was being index’ed.

The question you should really ask is "why on earth did I use Google 
first rather than Bugzilla search"?

Gerv