external searching bugzilla database, robots.txt

[Jason Remillard]

Hi,

Gervase Markham wrote:
> Jason Remillard wrote:
> 
>> Now that I have a clue, I see that there is a bug (81920) against not 
>> letting in search engines. It is marked as “will not fix”. I am not 
>> sure if it was marked as do not fix because letting in search engines 
>> was considered a bad idea, because the solution suggested in the bug 
>> was bad, 
> 
> 
> It's a bad idea - because the search engine will display out-of-date 
> information, and because having search engines crawling the site would 
> probably be a significant load that we can't bear.
>

The idea of the search is to see if something exists, and to find it. My 
specific example was attempting to understand the RSS support in 
bugzilla past present and future. I did not know if it was already 
implemented, I did not know if it was on a road map, I did not know if a 
patch existed, I did not know if it was already checked into the 
development branch, I did not know if it was being worked on by someone, 
etc. Normally, google would be the correct place to get this kind of 
information compared to the projects bug database (as a site note, I put 
a patch in for yesterday at 
http://bugzilla.mozilla.org/show_bug.cgi?id=82878 if anybody cares)

If the main bugzilla server is overloaded, then that is a separate issue 
from what is good for the bugzilla project. My guess is that 99% of the 
existing bugzilla installs are not overloaded. At work I need to *beg* 
to get people to use bugzilla database.


>> or because it was not going to be done for the current release (2.16).
> 
> 
> If that were true, its target milestone would just have been changed.
> 

OK.

>> Obviously I think that it is a good thing to unlock the wealth of 
>> information the bugzilla database to the search engines. Also, many 
>> companies have internal search spiders, so I think the value in 
>> indexing holds up for both public and private installations. I am sure 
>> it would have saved me time if the public bugzilla database was being 
>> index’ed.
> 
> 
> The question you should really ask is "why on earth did I use Google 
> first rather than Bugzilla search"?
>

The reason is that a reasonable person would expect things to be in a 
search engine is because every other site on the planet is falling all 
over themselves making sure they are index'ed. Why have a web site if 
you don't care if people can find your information? Like I said in the 
beginning, it did not occur to me that all of the interesting bugzilla 
development stuff would be hidden from view in your bugzilla database 
(rather than email lists, news groups, etc).

But anyway, you answered my main question. You don't want a patch for 
this feature. Correct?

No problem, I will just use the suggestion from Jason Pyeron about 
linking the ant image to the empty search for my local install rather 
than worrying about getting a patch in shape for 2.20.

Thanks
Jason.