backporting minor UI changes to keep QA scripts working

Frédéric Buclin LpSolit at gmail.com
Wed Jan 18 22:11:42 UTC 2006


> This means that, if you write a spiffy new test script to test something
> that's never been tested before, you'll probably just write it for the
> trunk. That's fine - Bugzilla's continually moving forward. And you can
> use the time you would have spent backporting it to write another cool
> test script.

Very funny. We are writing these tests for the first time, so we could 
hardly freeze together with the branch. What you say is to not test 
2.20, at least using automated tests. So why wasting our time doing QA 
tests on branches? Come on, let's release them tomorrow, untested.


> have wide coverage. But frankly, why run newly-written test scripts
> against 2.20? If you find bugs, are we going to fix them? Probably not,
> because it's a stable branch.

2.20 has been released ~ 3 months ago and will live for at least 2 
years. Everytime we want a new 2.20.x release, we will have to test it 
to make sure nothing is broken. And without these tests, this means 
doing the work manually again and again. Thanks a lot, but the QA team 
already did 3 such releases since July and we are very tired of doing 
this again and again... and it's very time consuming and require too 
many people (and I never got replies when asking help for QA stuff).



> What is the point of test scripts?
> 1) To prevent regressions during development before freeze
> 2) To prevent regressions during security fixing after freeze
> 
> 1) doesn't apply to 2.20. OK, they might be useful for 2), but we've
> done manual testing of security fixes for ages now, and it won't hurt us
> to have to keep going a bit longer.

Wrong! Also to prevent regressions due to "normal" fixes on branches. 
The reason we created the QA team last summer was because two releases 
were broken and we had to do emergency releases in both cases. None of 
these broken releases was based on tip, but both were based on branches 
(2.16.9 and 2.18.2 IIRC). And doing testing manually *does* hurt 
actually. It requires time and people. AFAIK, we are not a lot of people 
taking care of testing branches. If you are happy with manual testing, 
find someone else.

I'm trying to bring QA stuff out of the stone age, and for this, I 
require some tiny UI changes, read my previous message. If that's too 
much for you, by "freezing" the branch, I didn't know you were talking 
about zero Kelvin!

LpSolit



More information about the developers mailing list