On Scmbug

[Mark Reibert]

On Fri, 2004-06-04 at 14:11, Kristis Makris wrote:
> > idea. Welcome aboard :)
> 
> :)
> 
> Mark and John, feel free to jump in here.

Kristis et. al.,

I am obviously picking up the pieces of this thread, so I'll just kick
in some random thoughts as I read along.

> > It also solves one of the other big problems - mapping cvs usernames to
> > bz logins. Your solution is workable, but I think the correct way of
> > doing that sort of thing is by adding an extra field to the user table.
> 
> I do think its the right think to do. Where do you draw the line ? Will
> you add a one-to-many relation of files affected per bug and their
> version numbers ? What happens if gnats does not offers an extra field ?
> scmbugd can support either methods, or even do so according to the
> bugzilla version (what I have works for 2.14, but could be changed to
> behave differently for 2.20 if you add such a feature). What happens when some of us move a directory by hacking directly a CVS repository, and then decide they want that change reflected in bz. This a hack around a hack scenario, but I'm just throwing some ideas out there.

The only hope for versatility is not to require changes in the
change-request tool (such as name mapping, etc.). I realize the
necessity of something like this can be successfully argued under
certain scenarios, but it should be resisted at all costs.

The same can be written for the revision-control tool. The integration
glue should not be concerned with, for example, CVS repository hacks. If
you start down that path you will find there is no end to the vicious
cycle. (Of course, if you find the need to hack around the tool too
often then you should consider another tool! BTW, did you see the Samba
guys have moved to Subversion?)

> > Do you have a list of other bug trackers that you'd like to see
> > supported? The only one I can think of that is worth pondering is gnats,
> > but lets not give gnats users any more reason to stay with it ;)
> 
> Between you and me I agree, but I still want to keep this a generic
> "change management" activity with "logging" integration. This could be
> svn->bz, cvs->clearquest, rcs->blog or webpage, router config change ->
> bz, etc. Mark likes RCSing all configuration files in /etc, and I'm sure wouldn't oppose to some RCS to bz integration.

Hmmm, I want to see how you get RCS->bz integration w/o the ability to
trigger in RCS! (I suppose you could wrap all the client commands, but
who wants to maintain that?)

> > The other question is how many setups are actually going to have
> > multiple SCMs talking to one bug tracker (or even more unlikely - the
> 
> I don't know, but why limit people's options when we don't have to ? (I
> can swear I had a stronger argument for this). I'd like to add support
> for once scmbugd supporting multiple glue->bugtracking connections, so
> that one daemon can integrate cvs->bz for some products, and svn->gnats
> for others (if that's what your new high-paying client uses!).
> (http://bugzilla.mkgnu.net/show_bug.cgi?id=279)

Postponing for the moment the larger discussion on whether or not the
integration glue should run in a daemon, I don't see the need to have a
one-size-fits-all scmbugd. scmbugd is just a piece of the glue. You
still need glue between the revision-control tool and scmbugd (usually
in the form of triggers) and between scmbugd and the change-request tool
(in the form of some kind of code to map the scmbugd requests onto
requests the change-request tool understands). What do we gain by
running this for every combination of revision-control and
change-request tool in one instance of the daemon? Generally this sort
of centralization eases maintenance at the expense of flexibility,
reliability, and "robustness".

> > other way round)? And once you've solved the problem of a) getting the
> > data from the scm, and b) getting it into the tracker anything in the
> > middle that doesn't need to be there is simply potential maintainance
> > and support problems. 
> 
> Not quite simply. I've experienced such maintainance and support
> problems in the past by not having this middle man:
> 1 - setting the minimum characters of a log message is done once in the
> daemon, rather than in 17 glued repositories. Same for changing various
> checking policies, such as the status of the bug before you commit (may
> want to allow commits against "closed"  or "confirmed" bugs). Same for
> changing the format of your commit messages, and adding other daemon
> features (emulate atomic cvs commits, moin-moin autolinking, etc.)

As I wrote above, you lose flexibility in this sort of centralization.
Maybe I want different minimum-length comments and checkin policies for
each repository ...

> 2 - What happens when you setup your own cvs repository on your laptop
> for testing, but your laptop username does not match the one used in
> your LAN for bz ? Having this extra authentication layer would permit
> Joe Schmoe to authenticate against his custom text file, or through
> other mechanisms, or not at all (in testing  maybe ??)! 

Not another passwd file to maintain ...

> 3 - Extracting changelog information should happen through scmbugd
> (http://bugzilla.mkgnu.net/show_bug.cgi?id=80 - I'm still thinking about
> this). Windows users would not be prohibited from getting such logs,
> only because they are missing a bunch of Perl libraries, and neither
> should Linux users need some graph drawing libs in their local
> workstations to create a changelog and make a product release. Scmbugd
> should take solve this problem in a central location.

I need to think about this too, but my gut feeling is this should not be
the domain of the integration tool. However, I reserve the right to
change my mind!

> 5 - You can hide your mysql database behing port filtering on the same
> machine scmbugd runs, rather than directly hit it from any repository
> location.

In my perfect world the integration glue uses only the published API of
the change-request tool (web service, socket connection, etc.). Hacking
directly in a DB is just asking for problems.

> 7 - You get integration logs in /var/log/scmbug rather than look in sendmail or whathaveyou logs to see if your integration went through, and if not what the problem was.

Yeah, whatever ... I get (administrative) logs somewhere. The real point
is the client should be notified (by the means of the underlying
revision-control tool) if the commit failed. Developers work in a
revision-control tool - don't make anything that changes the implicit
expectations.

> > In the case of cvs->bz or svn->bz it gives you nothing. I also feel
> 
> It gives you a common source of upgrades (and everything mentioned
> above). scmbugd is also a common source for blocking development, but

Yeah, well killing my pserver or svnserve or SVN WebDAV is pretty good
at blocking development also!

> then so is bugzilla. At least you know that you are down, which is better than
> sending email without any notification of whether the bz commit
> succeeded. Notification from Scmbug is synchcronous. Worst case scenario, you can disable the glue. But at least
> you are conscious that you did that! It also permits scmbugd to solve some
> problems bz shouldn't have to deal with (and neither should gnats, or anybody else), such as cvs's lack of atomic
> commits.

CVS's lack of atomic commits will be rendered irrelevant when Subversion
replaces it as the de-facto standard for open-source projects. I am not
being religious here, I just can't see the world sticking with CVS when
there is a better tool available. (Of course, there is quite a bit of
inertia to overcome ...)

> > quite wary of having a long-running deamon process written in a
> > scripting language (been there, done that).

Performance may also be an issue under heavy loads.

> I needed this integration desperately, but frankly cvszilla, McIntere's
> work, and the email gateway just didn't work for me. Plus I could not reuse
> any of their source at all. Everybody assumed a certain schema, certain bz
> version, provided certain features I did not want and could not disable (cvszilla modified the
> bz schema!) or had some dirty code tangled up in blue. Lets solve this
> problem once and for all. Standardize on an integration protocol and
> interface. I was aiming for a svn -> bz integration (which is part of Mark's bigger plan for a change request tool), but wanted to get
> cvs working first, since that's what I was mostly familiar with and used
> John's integration before. Practically I just changed the design. The ideas for most of the goodies are his.

In terms of general comments, my interest is to create a SCM system from
the viewpoint of the revision-control tool whereas the approach here is
to tackle the problem in the eyes of the change-request tool. In an
ideal world I suppose the distinction is largely academic in as much as
all revision-control and change-request tools would speak the same
language. But in practice the vantage point can be important. For
example, I may want a "tagging" or "labeling" operation in my
revision-control tool to add a "version" into the change-request tool
(so that I can write bugs against that version). That works well if we
use something like CVS, ClearCase, etc., but is significantly more
difficult in, say, Subversion as "tags" are simply tree copies. The
point here is the intersection between the revision-control and
change-request tools varies with the products and can drive the design
of the integration code.

>From the technical side, why a daemon? As I mentioned above, scmbugd is
really the glue between the glue - it connects the revision-control
triggers to the change-request API. As such, there are a number of ways
this could be implemented. I am curious as to why you chose a daemon.

Some day I will send you a diagram of my high-level architecture ideas.
(Right after I work 50+ hours, mow the yards, install some misters on
the back patio, clean up the garage, replace the water main at my
parents house, spend some time with the wife so she doesn't leave me,
and finish trimming out the kitchen windows I installed over two years
ago. Yes Kristis, I still haven't done that! ;-)

Regards,
Mark Reibert

-- 

----------------------
Mark S. Reibert, Ph.D.
mark at reibert.com
----------------------