backporting minor UI changes to keep QA scripts working
Frédéric Buclin
LpSolit at gmail.com
Wed Jan 18 22:11:42 UTC 2006
> This means that, if you write a spiffy new test script to test something
> that's never been tested before, you'll probably just write it for the
> trunk. That's fine - Bugzilla's continually moving forward. And you can
> use the time you would have spent backporting it to write another cool
> test script.
Very funny. We are writing these tests for the first time, so we could
hardly freeze together with the branch. What you say is to not test
2.20, at least using automated tests. So why wasting our time doing QA
tests on branches? Come on, let's release them tomorrow, untested.
> have wide coverage. But frankly, why run newly-written test scripts
> against 2.20? If you find bugs, are we going to fix them? Probably not,
> because it's a stable branch.
2.20 has been released ~ 3 months ago and will live for at least 2
years. Everytime we want a new 2.20.x release, we will have to test it
to make sure nothing is broken. And without these tests, this means
doing the work manually again and again. Thanks a lot, but the QA team
already did 3 such releases since July and we are very tired of doing
this again and again... and it's very time consuming and require too
many people (and I never got replies when asking help for QA stuff).
> What is the point of test scripts?
> 1) To prevent regressions during development before freeze
> 2) To prevent regressions during security fixing after freeze
>
> 1) doesn't apply to 2.20. OK, they might be useful for 2), but we've
> done manual testing of security fixes for ages now, and it won't hurt us
> to have to keep going a bit longer.
Wrong! Also to prevent regressions due to "normal" fixes on branches.
The reason we created the QA team last summer was because two releases
were broken and we had to do emergency releases in both cases. None of
these broken releases was based on tip, but both were based on branches
(2.16.9 and 2.18.2 IIRC). And doing testing manually *does* hurt
actually. It requires time and people. AFAIK, we are not a lot of people
taking care of testing branches. If you are happy with manual testing,
find someone else.
I'm trying to bring QA stuff out of the stone age, and for this, I
require some tiny UI changes, read my previous message. If that's too
much for you, by "freezing" the branch, I didn't know you were talking
about zero Kelvin!
LpSolit
More information about the developers
mailing list