Self-Introduction: Fergus Sullivan
dmarshal at yahoo-inc.com
Wed Aug 29 19:49:08 UTC 2007
Note: I work for Fergus and will compose a self-introduction soon...
One of the things I asked about when deciding to come work for Yahoo!
was whether we would be sharing changes with Mozilla. The idea got a
very warm reception. The only reason we haven't been sharing is
because our code isn't quite up to snuff and there have been greater
priorities. We are finally starting to settle down into a steady
state of code joy, so there has been more recent discussion of how
and when to contribute.
On Aug 29, 2007, at 12:19 PM, Gervase Markham wrote:
> So much to ask about!
> Fergus Sullivan wrote:
>> == Anything else you'd like to say ==
>> Yahoo has been happily using Bugzilla for the last six years. We
>> now have a mandate from our top-level management to contribute to
>> the open source effort.
>> We are currently using a customized version of 2.22.
> Do you have an upgrade plan?
Yes, but it's slow going. (I have been at Yahoo! since January, so
my information may be slightly incorrect.) There have been competing
factions in the past that have pushed for either forking our Bugzilla
from b.m.o, followed by at least one effort to defork. We have
wrestled with this question at length, and what we want to do is to
make all the changes that made 2.22 -> 3.0 happen, but it's a large
>> We serve Bugzilla on FreeBSD 6, mysql 4.1, perl 5.6. We use a
>> mixture of MyISAM and InnoDB.
> As in, different tables use different engines? Which uses which?
Basically, everything that doesn't need a full-text index is in
InnoDB. We are exploring how to use InnoDB for all the tables.
>> We have a dual-master database (we have our own proxy that can
>> switch as needed)
> So MySQL supports this?
Er, no, it doesn't... :) read "We have our own proxy" again.
>> and a VIP handling 3 front-end and servers and search slaves. We
>> have a second colo for BCP with an equivalent architecture. We
>> have additional machines as reporting slaves.
> As in, they have read-only mirrors of the DB and are used for
> running reports?
Yes, we give RO-access to people who want to run their own reports.
It beats having to tell someone why we aren't going to add their
>> Our database currently stands at about 1.5 million bugs and grows
>> at about 50k per month.
> That's the biggest known Bugzilla database, by about 3x. GNOME is
> at about 470,000.
>> We have 20,000 named accounts (one per employee, plus many
>> internal mailing lists), of whom about 8,300 are active in any one
> LDAP? RADIUS? Or a mirror of your internal employee database into
> the Bugzilla DB?
We have several databases from which we extract information for our
>> In terms of active users and new bugs opened, our load has
>> increased 50% in the last year. We have 2,400 separate products
>> in our DB.
> Red Hat has 7814 components in a single product, but I don't know
> of anyone who has that many products.
>> In addition to myself, we have two developers and one QA assigned
>> full time on Bugzilla. We have another two guys working full time
>> on it at the moment, although they may move to other tasks in a
>> few months time.
> Presumably you have a testing installation?
>> - The core SQL within Bugzilla did not scale to meet our needs.
>> We reached a tipping point at around 800,000 bugs. The main cause
>> was lock contention within the DB. The causes and solutions would
>> fill an academic paper. We rewrote search.pm and greatly
>> increased performance. Some of the changes were trivial. A two
>> line change improved our "search for bugs where person X is cced"
>> from 45 seconds to 0.4 seconds. Other changes were vast. The
>> overall effect was search times have dropped from up to 45 seconds
>> in typical usage to about six seconds.
> I'm certainly looking forward to seeing these! Do you have any idea
> of how well the changes fit with our changes between 2.22 and now?
I think they will fit in reasonably well, although it remains to be
seen whether anyone who doesn't have a million rows in bugs would
notice any particular improvement.
> Was it only search that was a problem?
I have found a couple of places (unknown whether they're Mozilla or
Yahoo! code) that would gather information about products/components/
etc one row at a time with separate queries. For most installations,
it's probably not a noticeable performance hit to do this. For us,
it was a killer, and I have altered those to hit the database once.
>> Product Management
>> - With 2,400 different products, the vanilla product chooser does
>> not scale for us. We have built an entirely different version.
> Have you been eyeing Classifications with interest, or do they not
> help? How do you do the UI for this?
We use classifications, but I don't think it's well integrated in our
product-choosing workflow. (Although I'm not a typical Bugzilla
user, of course.)
>> - We have made many minor and a few major changes to the UI. One
>> side sorting of buglist results. This greatly reduces server load.
> I use a Greasemonkey script for this. I agree, it needs to be a
> core feature.
>> - We have unified comments and the bug activity table. Both
>> appear inline in the show bug page.
> Do you find this useful in practice?
This is still new, so we don't have a lot of feedback about it yet.
I think it will be very useful, especially as we allow users to
collapse this detail.
> To view or change your list settings, click here:
More information about the developers