Branch release problems

[Benton, Kevin]

> Tonight we released a 2.18.2 release with a non-functioning query
page.
> 
> I had to post an announcement telling people not to download it and
wait
> for an update.  That's not a fun thing to do.  This has also happened
> before (branch releases going out with major regressions in them) with
> versions 2.16.9 and 2.14.5.  This is really unacceptable and it messes
> badly with the public perception of our credibility.
> 
> We really need some kind of better QA process on our branch releases.
I
> know it's hard because all of the developers run the cvs tip or the
> developer releases for their own installations, so nobody's familiar
> with the branches, but that's what we have landfill for, and we need
to
> make sure things get tested before they go out.
> 
> At a minimum we need to put together some kind of smoke test document
> with lists of features to test for each branch, then we need to make
> sure that testing *everything* on that list is made part of the
release
> process, before a tarball is rolled, no matter how few patches
actually
> got checked in, or what they touched.
> 
> I'd welcome any other ideas anyone has, too.
> 
> We've also got almost zero actual QA since MattyT got busy with real
> life.  It'd probably be a cool thing to get some people to actually do
> QA type things for us again.  Bugzilla is growing by leaps and bounds,
> and we definitely have a big enough product to badly need a whole QA
> team, let alone the one-man-show that MattyT used to run.

Dave - that's why I'm actively working on a pre-release testing guide
for Bugzilla use here.  Part of what I'm doing, however, is making it so
I can produce the necessary documentation for general Bugzilla testing
as well.  The process of creating this guide is so involved, however,
that I don't see it being done for at least a few months.

---
Kevin Benton
Perl/Bugzilla Developer
Advanced Micro Devices
 
The opinions stated in this communication do not necessarily reflect the
view of Advanced Micro Devices and have not been reviewed by management.
This communication may contain sensitive and/or confidential and/or
proprietary information.  Distribution of such information is strictly
prohibited without prior consent of Advanced Micro Devices.  This
communication is for the intended recipient(s) only.  If you have
received this communication in error, please notify the sender, then
destroy any remaining copies of this communication.