Self-Introduction: Fergus Sullivan
fergus at yahoo-inc.com
Wed Aug 29 22:51:00 UTC 2007
Many thanks to each of you for the very kind welcome. We're looking
forward to working with you guys.
There are a couple of open questions that we haven't yet covered, so
there's more info below.
== How do we have dual-master sql proxies? ==
We have our own secret sauce for this. It's in use across Yahoo and
is central to our BCP/HA planning. It's proprietary, and not
currently on our list to release to open source. I have, however,
prodded its owners about that. Essentially it allows us to switch
from Master A to Master B the moment Master A fails a health check.
We put the two masters in different server farms so as to reduce
== What is BCP/HA? ==
Business Continuity Planning is a means to ensure that if our primary
colo goes down we can switch to another as needed. It assumes a
certain amount of downtime as dns records are switched and boxes are
spun up. This could be anywhere from 30 minutes to a couple of hours.
High Availability means that the service should never go down, and
automatically switch to a backup server farm as needed.
In both cases, the systems exist to allow for natural disasters,
power outages, war, insurrection, comets or acts of god. Read your
insurance policy's exclusion policy for more...
In both cases, data in each location needs to be up to date.
== How many Bugzilla servers does Yahoo have ==
A couple of years ago we had 1. Then we bumped up to three.
Currently we're at one db master and three slaves which share the
http and search load. These sit behind a load-balancer.
Our next hardware revision will bump us to one master and four slaves
in each of two server farms. One farm is on the US west coast, the
other on the east. The primary goal of this is HA, not server load.
We are evaluating the need for mirrors local to our users in Asia and
We have one read-only mirror where we let users run their own raw sql
And we have "The Bug Hole", which is where your query is sent if it
requires a full table scan. This type of query was blocking other
users and hogging our search resources.
Incoming mail has always been a part of our instance and shares one
of the other servers.
== What's this YUI thing, and why will Ferg be harping on about it
for the next six months? ==
YUI is the Yahoo User Interface libraries. These were released under
the FreeBSD license a year and a half ago. They are widely used,
both at Yahoo and outside. Airline sites like southwest.com and
northwest.com make heavy use of them.
The YUI libraries include widgets like datatables, grid management,
dialog management, CSS levelling, trees, drag and drop etc. They are
fully functional cross-platform and cross-browser.
I joke with the YUI guys that their work makes bad engineers look
good. My point is that a little time with the libraries can make the
trickiest tasks easy. For example, we decided to pipe the content of
buglist.cgi into a datatable so that we could sort it client-side.
It took one of our guys two days to get it working.
The YUI team sit across the aisle from us. When we need a new
feature, we wander over and ask. We dreamed up the idea of how cool
it would be to be able to edit a bug list in place, like an excel
sheet. We walked over to the YUI guys and asked them if it could be
done. And lo, inline editing in the datatable was born.
Here are some of my favorite examples, and how we could use them in
Datatable - inline editing of the buglist. Browse around to see
examples of client-side sorting and pagination goodies. Don't
neglect the client-side pagination with browser history.
TreeView - Dependencies anyone?
Calendar and Autocomplete - We already use these in our Bugzilla code
Grids - For clean layout
There are many, many other uses besides. For us, use of the YUI
libraries has been the best thing ever. The only catch bugzilla.org
needs to consider is that if we all go with YUI we'll never want to
== Test harnesses ==
As dwm mentioned, we have a suite of tests that use an internal Yahoo
tool. We try hard to keep our tests up to date with a 100% pass rate.
We have a number of test machines. These are mainly for load testing
in the form of log replays.
== The Yahoo Product Chooser ==
With 2,500 products we needed an optimized product chooser. To be
honest, I'm not at all happy with its current form, so we may well
change it in the near future. We're thinking of something analogous
to an address book module in a mail client. Essentially we want to
make it easier to select one or many products, and steer people away
from searching 'all'.
== When can Yahoo start offering its patches for review ==
It depends. We have an internal legal process to approve
engagement. That'll take a few weeks to complete. We should be able
to offer minor patches soon after.
The larger patches will be tricky, as they work for us right now with
the rest of our code base. Remember our code is based on 2.22 and
chunks of it are modified. That said, our Search.pm is a thing of
More information about the developers