Results of Community Research

[Max Kanat-Alexander]

	Hey folks. So, as you know, I sent out a message a few weeks ago asking
former contributors why they left. Also, as you may not know, before
that, for several days I was involved in a statistical analysis of the
history of Bugzilla contributions. Basically, I looked at how many
different contributors we had in CVS for every 30 day period since the
start of the Bugzilla Project. I wrote a script that gave me the data by
parsing out who the "Patch By" email address was, or I used the
committer if there was no "Patch By" in the commit message, and I joined
people who I knew to be duplicates, which were very few.

	The resulting graph was very interesting. It showed two interesting things:

	* Freezes kill the community--every time we had a code freeze, the
number of contributors would dramatically drop and then recover slowly.

	* Despite this, the size of the community generally increased until the
start of 2005, and then leveled off through 2005, and then started to
slowly decrease after that. No matter how I reorganized the data, or how
I analyzed it, 2005 was the critical year.

	So my question was--what happened in 2005 that changed things so
dramatically? After QUITE a bit of searching, I discovered the following:

	1) The "review" and "review-requests" mailing lists were split at the
start of 2005. Before, all reviewers had at least seen every single
review request. Now they had to subscribe to review-requests and
actually *request* all that spam, which nobody was going to do. (I
didn't even know about the review-requests list by the time I became a
reviewer, which was after this point.) I think that this particularly
hurt people's ability to get review "from the wind", since only old-time
and very dedicated reviewers would have been on the list where those
requests went.

	2) We froze *twice* for 2.20.

	#2 wouldn't have been so bad, and we would have recovered, I think, if
not for #1. But why would that review-requests change have such a
profound effect? Well, the answers to that were at least partially in
the survey results. (By the way, I really appreciate all the answers
that I got, they were extremely helpful.)

	The #1 reason that people leave is simply that they no longer have time
to contribute, or that they were contributing as part of their job and
now they have changed jobs. The survey data showed that it is to some
degree inevitable that contributors are going to leave.

	I'd like to note here that nobody gave "the review process is too hard"
as a reason for leaving. Some people here and there have banged the drum
for a lowering of quality as an incentive to attract and retain
community members, and there was absolutely no evidence in the survey
that that would improve the situation at all.

	Both of the former documentors who responded mentioned the lack of
kudos from the community as one reason that they weren't motivated to
stay in the community. Several other people mentioned negative
interactions or simply a lack of appreciation as part of the reason that
they left.

	In fact, for nearly every person who responded, the factors involved in
not staying, beyond "my job changed" or "I didn't have time", were
surprisingly personal. I know that we all work with computers, and
perhaps we'd like to think that engineering should be a totally cold
scientific profession where we all do our jobs correctly according to
the requirements of the machine, and not worry about our emotional or
personal involvements. But nothing could be further from the truth--the
personal interactions that people have with community members, the
amount they feel appreciated, and the amount they feel assaulted, are
actually the MOST IMPORTANT ASPECTS OF RETAINING COMMUNITY MEMBERS.

	And *retaining* community members is what we have to do, particularly
*new* community members. Because the statistical analysis and the survey
both showed that the eventual leaving of many contributors is
*inevitable*. What we have to focus on is locating, accepting, and
encouraging *new* contributors. If we don't continuously *grow* the
community with new members, it will continuously shrink as old ones
leave, no matter what else we do.

	The survey tells me that the keys to retaining new community members are:

	1) Extremely rapid response to review requests. No matter how much
review a contributor has to go through, they usually will go through it
as long as there aren't LONG DELAYS between the request and the response.
	
	2) Extreme kindness and visible appreciation. When people contribute on
a volunteer basis, they aren't getting paid in money, they are getting
paid in admiration, appreciation, the sense of a job well done, and the
knowledge that they are helping create a product that affects millions
of people directly and even more people indirectly (because people use
Bugzilla as part of making other products). When somebody has
contributed a patch, WE NEED TO THANK THEM. It doesn't matter if the
patch is total crap and has to be re-written entirely, WE NEED TO THANK
THEM. They have put some work into this, and if we don't appreciate
that, THEY WILL LEAVE BEFORE THEY EVEN START. After all, most people get
little enough appreciation at their workplace--they stay there because
they get paid in money! They don't need to work FOR FREE with some other
organization if it also doesn't appreciate their work, or even worse,
assaults every aspect of their contribution before even thanking them
for it.
	This isn't to say that we can't correct people on the faults in their
patches. "Kindness" does not include putting bad code into Bugzilla.
That isn't kind to anybody, including the contributor whose skills
probably need to improve, and who may go on believing that something
they did in error was in fact correct. We have to still be careful
reviewers and very good coders.
	What this does mean is that in addition to telling people what's
*wrong* with their code, it's important to appreciate what's *right*
about their contribution, even if it's simply the fact that they took
the time to contribute. And you have to ACTUALLY TELL THE CONTRIBUTOR
THAT YOU APPRECIATE THE CONTRIBUTION. The more frequently and genuinely
that all of us do this, the more likely we are to retain the contributor.
	Some people are hopeless programmers, and they're never going to get
better no matter how much you help them or how many reviews you do. For
these people, you don't have to imagine nice things to say about them--I
don't expect anybody to ever make up stuff that isn't true, just to be
nice. If somebody has demonstrated conclusively that they're never going
to get any better, or that they're just going to cause trouble one way
or another no matter what we do, the best thing you can do is simply
personally choose to ignore them, which is a personal decision that you
can make for yourself.
	But that represents a very, very small percentage of total
contributors. For the vast majority of contributors, even though they're
a little rough around the edges for the first few months, the correct
action is to be helpful, kind, appreciative, and responsive.

	3) A total absence of personal negativity. One thing that drives people
away from a project with lightning speed is when they get personally
attacked for attempting to do something positive. A "personal attack"
can be as little as an unpleasant joke about their code, instead of just
a straightforward technical description of what is wrong. Saying
something like, "What is wrong with you?" instead of actually leaving
some helpful comment. Disguising personal criticism as "an attempt to
help them code better" or "help them get along with others". (If
somebody does something that offends you, just tell them that it
offended you--that's usually totally enough to resolve the problem. You
don't have to pretend that other people have a problem just because you do.)
	Look, I understand that sometimes coding and working on a collaborative
project with people who have different viewpoints can get REALLY
frustrating! I've been an offender in this area just as much as anybody
has been! But it's really not OK for us to insult other developers
personally just because we're frustrated with them personally.
	If you're having a bad day, then please feel free to take a break.
Getting angry or upset with other developers in a personal way could
actually be more damaging to the long-term future of the Project than a
single day's worth of contributions would help.
	And finally, if there's some contributor that you just can't live with,
please feel free to come to me, and I will absolutely do my best to work
it out, even if it's been a really long-term thing! I'm all about being
effective, too--I'm not going to just brush it off and say "well, maybe
it all will fix itself," I'll do something effective about it. And if
it's *me* that you have a problem with, please go to justdave about it,
and he'll talk to me about it--he's very good at that sort of thing. :-)
	The most important thing is that you don't take it out on the person
you're unhappy with--I don't think that that helps anybody, and it
definitely harms the long-term future of the Project.

	Now, #2 and #3 above pretty much cover my recommendations on how we
should behave toward new community members: be really, abnormally,
really, really kind, and don't be mean. But how do we address the
problem of the delay on reviews?

	Well, here's my recommendations for how we handle the review problem:

	1. I personally have started to prioritize review requests from
non-core contributors above anything else, including security work,
releases, ANYTHING. New contributors are my #1 priority, and I've
already seen this have a big effect on the number of contributions we're
getting from outside developers.

	2. I think we should say that if you're a reviewer, it's your
responsibility to receive the review-requests emails, and we should
merge review-requests back into review at . You don't have to have them go
into your inbox, but you should at least receive and read them. I think
that this would get a lot of people more involved in review, because
everybody would get every review request, and even review from the
"wind" might start to work again. Particularly for the less-active
reviewers, this would give them more ability to pick and choose a few
items that they wanted to review now and then, and it would also let
each reviewer know if there was some area of their interest under review
so that they could chime in on it.

	So whew! That's the results and my recommendations! :-) What do you
guys think? :-)

	-Max
-- 
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.