From gerv at mozilla.org  Fri Aug  9 20:52:05 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Fri, 09 Aug 2002 21:52:05 +0100
Subject: Module naming
Message-ID: <3D542B75.2080402@mozilla.org>

We are going for Bugzilla::Foo.pm , unless anyone screams very loudly 
very soon.

Gerv



From gerv at mozilla.org  Sun Aug 11 15:24:10 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Sun, 11 Aug 2002 16:24:10 +0100
Subject: Localisers Guide
Message-ID: <3D56819A.2060202@mozilla.org>

Here's the first draft of a Bugzilla Localisers Guide. Comments welcome.

Gerv

<snip>

Bugzilla Localisers Guide

This guide outlines how to make a localised copy of Bugzilla.

Overview
--------

The key technology for localising Bugzilla is UI templates. Over the 
last few months, more and more of Bugzilla has been converted to use 
templates, and so now Bugzilla's user-facing UI is all done. This 
separates the content from the presentation.

We plan to templatise the administration UI as soon as we can but, in 
the mean time, you have to administer Bugzilla in English. Error 
messages are also not done in 2.16.

We recommend that you begin by localising Bugzilla 2.16. Although it is 
not fully localisable, it is still localisable enough for day-to-day 
use, and your localisation will form a good basis for a localisation of 
a later version in the future. In other words, by the time you've done 
that, we should have a better version ready for you to migrate to :-)

Doing the Localisation
----------------------

I am assuming you are creating a localisation for a country code "xx". 
You need to create a directory template/xx/default under the Bugzilla 
root, and recursively copy the contents of template/en/default into it.

Note that you can create a localisation for a two-part locale xx-YY - 
e.g. "en-GB" - but it will only be made available to people who 
specifically request "xx-YY"; people who request "xx" won't get it. If 
you do create an "xx-YY" localisation, make sure you have an "xx" 
localisation also.

Once you've got a copy of the templates, start translating the English 
in them. It should be obvious what to translate and what not to 
translate. Template toolkit directives and variables can be found inside 
[% %] tags - don't translate these. You can, however, move them around 
within a sentence for localisation purposes.

Also, some times are formatted in the template using time2str - if this 
is true, you can rearrange the time format to suit your locale.

This may take a significant amount of time - there are over 200 
templates. You may wish to share this work among multiple contributors.

Creating a Localisation Pack
-----------------------------

To create a localisation pack, tar up the contents of 
template/xx/default and make it available. Remember to mark it with the 
version number of the Bugzilla it corresponds to.

Using A Localisation
--------------------

To install a localisation pack, untar it into your Bugzilla directory so 
it appears in template/xx/default/ .

2.16: To get Bugzilla to use the localisation, you have two options. If 
you want Bugzilla always to be in that language, edit globals, pl, find 
INCLUDE_PATH and change the "en"s to "xx"s. If you want Bugzilla to pick 
up the user's browser language and use that, apply the patch from bug 
126955, and set the relevant parameters (lang and defaultlang) 
appropriately.

2.17 onwards: The patch from bug 126955 is already checked in - so just 
set the parameters to define what languages you want to use in what 
order. You can force use of a particular localisation by setting it as 
the default and giving no other options.



From bbaetz at student.usyd.edu.au  Sun Aug 11 23:55:38 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Mon, 12 Aug 2002 09:55:38 +1000 (EST)
Subject: Localisers Guide
In-Reply-To: <3D56819A.2060202@mozilla.org>
Message-ID: <Pine.LNX.4.44.0208120948370.1288-100000@tomato.home>

How doed non-ascii (well, non ISO-8859-1) text work? Comments on the TT
list suggest that that needs perl 5.8, but I havne't tested.

The TT Template::Plugin::Date plugin can be passed a locale param in TT 
v2.08. That uses the POSIX stuff, though; we may want to use Date::Format 
instead.

Bradley



From burnus at gmx.de  Sun Aug 11 16:50:45 2002
From: burnus at gmx.de (Tobias Burnus)
Date: Sun, 11 Aug 2002 18:50:45 +0200
Subject: Localisers Guide
References: <3D56819A.2060202@mozilla.org>
Message-ID: <3D5695E5.F1EDBD64@gmx.de>

Hi

Gervase Markham wrote:
> I am assuming you are creating a localisation for a country code "xx".
This is not a country code but a ISO 639[-2|3] language code
(http://www.loc.gov/standards/iso639-2/).

> You need to create a directory template/xx/default under the Bugzilla
> root, and recursively copy the contents of template/en/default into it.
Maybe also explaining the difference between custom and default?
In general one can assume that one creates a local bugzilla installation
for which the general templates is a byproduct.

> Note that you can create a localisation for a two-part locale xx-YY -
> e.g. "en-GB"
That is a RFC 1766 language tag. Maybe a "(RFC 1766)" after xx-YY would
be good.
(Standard references are ususally nice to check what tags have to be
used. Unless they are too long or used to extensively used since then
they clutter the complete text and thus unreadable.)

> Also, some times are formatted in the template using time2str - if this
> is true, you can rearrange the time format to suit your locale.
Which doesn't solve the localisation problem of the names of the
week/month.
(See
http://groups.google.com/groups?&selm=3D46BBB7.2090406%40physik.fu-berlin.de
and
http://groups.google.com/groups?selm=ai6pu3%24bfv3%40ripley.netscape.com)

One can also mention the trick which enables to translate hard-coded
strings.
+[% ordersdesc = {
+   "Reuse same sort as last time" => "zuletzt verwendeter Reihenfolge",
+   "Bug Number"   => "Fehlernummer" %]

-     [% $order FILTER html %]
+     [% ordersdesc.$order FILTER html %]

Tobias
-- 
This above all: To thine own self be true / And it must follow as
the night the day / Thou canst not then be false to any man.


From justdave at syndicomm.com  Sat Aug 17 20:34:22 2002
From: justdave at syndicomm.com (David Miller)
Date: Sat, 17 Aug 2002 16:34:22 -0400
Subject: "rename integrity" on the activity table?
Message-ID: <p05111a00b98462d5e7ba@[192.168.1.203]>

OK, an interesting point was brought up in the group changes stuff...

What are we doing with existing groupset entries in the bugs_activity table?

I'd assume we convert them over so they look like the current format.  But
do they keep names or IDs?

And if you keep IDs (which would make stats easy) then what happens if you
delete a group and the ID gets reused later?

This of course lends itself to products and components and so forth as well
(names versus IDs)

And if you rename something do all the activity entries for it get renamed
or do we make an entry in the activity table for each affected object
showing the rename taking place?

There was more to it than this but I thought I'd get the discussion going
at least.
-- 
Dave Miller    justdave at syndicomm.com + justdave at justdave.net
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.justdave.net/


From bbaetz at student.usyd.edu.au  Sun Aug 18 02:27:15 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sun, 18 Aug 2002 12:27:15 +1000 (EST)
Subject: "rename integrity" on the activity table?
In-Reply-To: <p05111a00b98462d5e7ba@[192.168.1.203]>
Message-ID: <Pine.LNX.4.44.0208181225240.1283-100000@tomato.home>

On Sat, 17 Aug 2002, David Miller wrote:

> OK, an interesting point was brought up in the group changes stuff...
> 
> What are we doing with existing groupset entries in the bugs_activity table?

Leaving them

> 
> I'd assume we convert them over so they look like the current format.  But
> do they keep names or IDs?
> 

Nope.

> And if you keep IDs (which would make stats easy) then what happens if you
> delete a group and the ID gets reused later?

Right - we need to keep the groupset stuff in there for exactly that 
reason.

> 
> This of course lends itself to products and components and so forth as well
> (names versus IDs)
> 
> And if you rename something do all the activity entries for it get renamed
> or do we make an entry in the activity table for each affected object
> showing the rename taking place?
> 

We leave it as is. We really have to do that, else we have to start 
parsing cc chang entries for email addresses, and so on.

If we change the activities table, then that means that mail will end up 
being sent, which is relaly not what we want. (Well, it might be, but 
thats a spoearate issue)

> There was more to it than this but I thought I'd get the discussion going
> at least.
> 

Bradley



From bugreport at peshkin.net  Sun Aug 18 04:44:11 2002
From: bugreport at peshkin.net (Joel Peshkin)
Date: Sat, 17 Aug 2002 21:44:11 -0700
Subject: "rename integrity" on the activity table?
References: <Pine.LNX.4.44.0208181225240.1283-100000@tomato.home>
Message-ID: <3D5F261B.3090201@peshkin.net>

There are a few problems with just leaving the old activity entries when 
the schema changes.  I recognize, though, that it is hard to say how far 
to go with keeping integrity later.

Let's start with the problem that prompted this.   As groupset goeas 
away, it becomes very bad to query on the now non-existant field. 
Search.pm takes everything in fielddefs and makes it searchable, so the 
natural way to get it out of the search form is to kill off the 
fielddefs record.

Unfortunately, this would leave a whole bunch of bugs_activity records 
dangling, not to mention that the activity record would have a suddent 
discontinuity in it where the schema conversion occurred.  So, I 
implemented an update in checksetup.pl that converts the existing 
bugs_activity records to match the new schema and then deletes the 
unused and absent field from fielddefs.  This is fairly easy.

However, this starts an interesting precedent (which would be very 
important to anyone using the activity log to generate reports) where we 
would expect the activity log to accurately reflect the activity even 
across more structural changes.  I think that this is a good thing to do. 
It would be fairly simple to add a RenameFieldVal(fieldname, oldval, 
newval) that could be called for things like 
RenameFieldVal("component","oldname","newname") and fairly simple to get 
all the rename-type code to do properly.

-Joel









From bbaetz at student.usyd.edu.au  Sun Aug 18 13:49:37 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sun, 18 Aug 2002 23:49:37 +1000 (EST)
Subject: Using DBI methods
Message-ID: <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home>

This got kind of long, but do read it all carefully.

In bug 163290, I'm moving the various DB stuff into a separate file, 
Bugzilla/DB.pm.

For the moment, this is being transferred fairly literally into an OO 
infrastructure.

However, it seems to me that it may be worth being able to use the DBI
methods directly. |perldoc DBI| lists those, for the interested.

Advantages:

1) Can use stuff like:
  my ($a, $b) = $dbh->selectrow_array("SELECT a,b FROM foo WHERE 
                                       prim_key=7");

or selectall_arrayref, and so on

which is nicer/cleaner/faster/easier to read than our sendsql+fetch stuff

2) We get to use DBI's error handling + exceptions - currently we only 
check for errors in one place, which is bad.

Note that all of these can reasonably easily be added to the current 
setup, but it starts to become less worthwhile/more complicated/etc as we 
go on

3) Can be faster - by using the routines as written in C in the DBI/DBD
  - unlikely to be really noticable since we don't tend to move large 
volumes of data to/from the db, but every bit helps.

4) No more state stack
  - don't have to worry about forgetting to push/pop the state stack -
simply get a new statement handle when you need one from teh globally 
available dbh when needed.

Disadvantages:

1) New syntax required
  - We can (and will) keep the old syntax arround for backwards 
compatability, though, for the reasonably future.
  - Mixing FetchOneColumn() and fetchrow_arrayref (for example) on ehte 
same statement handle won't work, because of the internal buffer 
MoreSQLData needs to determine emptyness.

2) Its slightly messier
  - Need to keep statment objects arround explicitly, rather than 
implicitly

3) We need to up the required DBI version
  - The currently required version dates to July 1999 (1.13). To use the 
HandleError stuff to do custom SQL error messages, we need 1.21 (Feb 2002)
  - Its also needed to subclass DBI, which is what we need for taint 
checking, and supporting the old (current) syntax (speaking of which, I 
should email the dbi list about the taint stuff...)
  - It has a lot of the cross-db stuff for transactions which we will need 
at some point
  - DBI-1.21 comes with debian stable and RH7.2/7.3 at least. Its not that 
hard to install it though, anyway

  - If DBI does add a 'TaintInOnly' mode, then we'll require that at some 
point, although probably not right away (or even at all for 2.18)

4) we need to up the required perl version to 5.005_03
  - that DBI version requires 5.005_03. We've been talking about it for a 
while, and 5.8 is now out, so requring the latest 5.005 release shouldn't 
be an issue, although I know that afranke will disagree...
  - Do note that the latest DBI version has a note that it may not support
5.005 for much longer. This may become an issue for us at some point in
the future.

What do people think about the above?

'Feature' removal:

I also plan to remove the following two features:

a) SQL log should indent the log as statements are pushed to the stack
  - If we no longer have a stack and have sth's which can be finshed in 
    an order different to when they were started, this is a pain to do.
  - The results don't really look any better, because very very very few of
    our SQL statements fit into 80 columns, let alone 80-8*depth. so it
    wraps anyway
  - More importantly, this doesn't actually work. It took me 15 minutes to 
    work out why not, and it hasn't worked since it was checked in almost 
    two years ago. Why? Because the case in |SavedSQLStates| is wrong.
    Why didn't this trigger a 'used only once' warning? Because it got
    added to the silliness stuff in a warning fix-up (bug 72862) :)

b) In theory, Param("shutdownhtml") will currently work even for scripts 
which don't |require 'CGI.pl'|.

Pre-templatisation/2.16, the check was in PutHeader, which was too 
late (and not called for non-CGIs anyway) Now, its in CGI.pl's toplevel 
stuff.

The current behaviour of having MoreSQLData() return 0 is a pain to 
re-implement, plus its buggy, and may cause stuff to happen which 
shouldn't (eg what if one of these calls is trying to regenerate the 
versioncache, or its a security check, or something?)

(I just checked, and regenerating the versioncache with shutdownhtml set 
does result in one which is empty.)

Eventually this will happen somewhere apart from CGI.pl, but for the
moment... This will cause different behaviour in non-cgi stuff, but I
consider an error (which Bugzilla::DB::connect will generate) to be 
preferable than generating bogus data/empty versioncache/etc

I'd be incredibly surprised if any of those programs work with shutdown 
html anyway - collectstats, for example, dies when it can't read from the 
product table a product which is in the versioncache.

(I'm tossing up between |connect| returning undef, dieing, or throwing an 
exception, and leaning towards the exception, which will basically mean a 
die)

If you object to the removal of any of the above please let me know ASAP.

Bradley



From afranke at ags.uni-sb.de  Sun Aug 18 15:41:23 2002
From: afranke at ags.uni-sb.de (Andreas Franke)
Date: Sun, 18 Aug 2002 17:41:23 +0200
Subject: Using DBI methods 
In-Reply-To: Your message of "Sun, 18 Aug 2002 23:49:37 +1000."
             <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home> 
Message-ID: <200208181541.RAA13047@ags.uni-sb.de>

> However, it seems to me that it may be worth being able to use the DBI
> methods directly. |perldoc DBI| lists those, for the interested.
> 
> Advantages:
> 
> 1) Can use stuff like:
>   my ($a, $b) = $dbh->selectrow_array("SELECT a,b FROM foo WHERE 
>                                        prim_key=7");
> 
> or selectall_arrayref, and so on
> 
> which is nicer/cleaner/faster/easier to read than our sendsql+fetch stuff

I think having short-cuts to get a single value, a single row, 
a single column etc. from the database is worth supporting; 
I've been using the following things for my application (which
doesn't have anything to do with perl or DBI, but I _am_ using
MySQL):
    
        default (this was the original generic behaviour; 
                 it returns a pair: 
                 1. a list of column names, and
                 2. a list of rows, 
                    where each row is a list of col values)
	records (perl equivalent: a list of hashes)
        tuples  (perl equivalent: an array)
        oneRow  (perl equivalent would be a list of values, 
                 one for each column; an exception is raised
                 if the result contains less or more than 
                 exactly one row)
        oneColumn (perl equivalent would be a list of values,
                   one for each row; an exception is raised
                   if the number of columns in the result
                   is not exactly 1)
        oneValue (returns the value; requires that the result
                  is exactly one row with exactly one value;
                  otherwise an exception is raised)

These values are given to my execSQL method (which corresponds to
SendSQL together with the loop fetching the result rows) as a 
'format' attribute. Note that my glue does not allow lazy fetching
of rows from the database yet (which I consider a bug in the glue
code), so this way of operation is missing from the above list.

This is not a proposal to introduce all these options for Bugzilla, 
I just want to say that I think it's a good thing to have the option
for multiple result formats, besides the generic SendSQL/fetch rows
stuff.

> 3) We need to up the required DBI version

> 4) we need to up the required perl version to 5.005_03
>   - that DBI version requires 5.005_03. We've been talking about it for a 
> while, and 5.8 is now out, so requring the latest 5.005 release shouldn't 
> be an issue, although I know that afranke will disagree...
>   - Do note that the latest DBI version has a note that it may not support
> 5.005 for much longer. This may become an issue for us at some point in
> the future.

I think requiring a recent version of DBI is not much of a problem,
and if this means that some perl versions cannot be supported any more,
so the perl version requirement needs to be up'ed. 
Just put a visible note in the release announcement and the upgrade web
page that those who are running old perl and are unwilling or unable
to upgrade that, should _not_ upgrade their bugzilla anymore until
that they have a better perl. A "last working cvs datestamp" would
be good to know as well.

> What do people think about the above?
> 
> 'Feature' removal:
> [...] 
> If you object to the removal of any of the above please let me know ASAP.

Looks like a reasonable cleanup to me. Thanks for doing all that work.

Andreas




From justdave at syndicomm.com  Sun Aug 18 21:03:44 2002
From: justdave at syndicomm.com (David Miller)
Date: Sun, 18 Aug 2002 17:03:44 -0400
Subject: Using DBI methods
In-Reply-To: <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home>
References: <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home>
Message-ID: <p05111a12b985bbc0bebd@[192.168.1.203]>

On 8/18/02 11:49 PM +1000, Bradley Baetz wrote:

> However, it seems to me that it may be worth being able to use the DBI
> methods directly. |perldoc DBI| lists those, for the interested.

> which is nicer/cleaner/faster/easier to read than our sendsql+fetch stuff

The only reason I know of that we HAVE to use SendSQL is to do the query
mirroring for the shadow database.  And we all agreed that the shadow
database was going away, right?

-- 
Dave Miller    justdave at syndicomm.com + justdave at justdave.net
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.justdave.net/


From joel at peshkin.net  Sun Aug 18 04:38:46 2002
From: joel at peshkin.net (joel)
Date: Sat, 17 Aug 2002 21:38:46 -0700
Subject: "rename integrity" on the activity table?
References: <Pine.LNX.4.44.0208181225240.1283-100000@tomato.home>
Message-ID: <3D5F24D6.2020506@peshkin.net>


There are a few problems with just leaving the old activity entries when 
the schema changes.  I recognize, though, that it is hard to say how far 
to go with keeping integrity later.

Let's start with the problem that prompted this.   As groupset goeas 
away, it becomes very bad to query on the now non-existant field. 
 Search.pm takes everything in fielddefs and makes it searchable, so the 
natural way to get it out of the search form is to kill off the 
fielddefs record.

Unfortunately, this would leave a whole bunch of bugs_activity records 
dangling, not to mention that the activity record would have a suddent 
discontinuity in it where the schema conversion occurred.  So, I 
implemented an update in checksetup.pl that converts the existing 
bugs_activity records to match the new schema and then deletes the 
unused and absent field from fielddefs.  This is fairly easy.

However, this starts an interesting precedent (which would be very 
important to anyone using the activity log to generate reports) where we 
would expect the activity log to accurately reflect the activity even 
across more structural changes.  I think that this is a good thing to do.  

It would be fairly simple to add a RenameFieldVal(fieldname, oldval, 
newval) that could be called for things like 
RenameFieldVal("component","oldname","newname") and fairly simple to get 
all the rename-type code to do properly.

-Joel







From bbaetz at student.usyd.edu.au  Mon Aug 19 01:11:39 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Mon, 19 Aug 2002 11:11:39 +1000 (EST)
Subject: Using DBI methods
In-Reply-To: <p05111a12b985bbc0bebd@[192.168.1.203]>
Message-ID: <Pine.LNX.4.44.0208191110450.10454-100000@tomato.home>

On Sun, 18 Aug 2002, David Miller wrote:

> On 8/18/02 11:49 PM +1000, Bradley Baetz wrote:
> 
> > However, it seems to me that it may be worth being able to use the DBI
> > methods directly. |perldoc DBI| lists those, for the interested.
> 
> > which is nicer/cleaner/faster/easier to read than our sendsql+fetch stuff
> 
> The only reason I know of that we HAVE to use SendSQL is to do the query
> mirroring for the shadow database.  And we all agreed that the shadow
> database was going away, right?
> 

Well, yes. But If I subclass the DBI,which I'll need to do anyway, then I 
can do the shadowlog mirroring in |execute|

Bradley



From bbaetz at student.usyd.edu.au  Mon Aug 19 06:18:43 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Mon, 19 Aug 2002 16:18:43 +1000 (EST)
Subject: Webtools:: vs Bugzilla:: packages
Message-ID: <Pine.LNX.4.44.0208191612440.4255-100000@tomato.home>

See bug 163291. In his review (on IRC), mattyt said that he thought that 
since this particular package wasn't bugzilla specific, it should go 
somewhere more generic.

My feeling is that while this is great in theory, in practice there'll
just be that one package with really generic functions (and perhaps one
more) which isn't bz specific - the others will be bugzilla-only.

Further, the chances that Bonsai will ever end up using any sort of
generic stuff we do is really really remote, not to mention that if it
ever does we'd have to keep backwards compatability in there somewhere to 
avoid breaking it.

IOW, its a great idea in theiry, but in practice theres no point.

I think that we can revisit this if, at some point in the future, someone
feels like maintainting bonsai.

Bradley



From gerv at mozilla.org  Mon Aug 19 06:42:32 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Mon, 19 Aug 2002 07:42:32 +0100
Subject: Webtools:: vs Bugzilla:: packages
References: <Pine.LNX.4.44.0208191612440.4255-100000@tomato.home>
Message-ID: <3D609358.5030808@mozilla.org>

Bradley Baetz wrote:
> IOW, its a great idea in theiry, but in practice theres no point.

I agree. And in the unlikely case of Bonsai using it, it wouldn't be a 
disaster if it used it out of a package called "Bugzilla", either, if we 
couldn't be bothered to move it.

Gerv



From preed at sigkill.com  Mon Aug 19 08:03:00 2002
From: preed at sigkill.com (J. Paul Reed)
Date: Mon, 19 Aug 2002 01:03:00 -0700 (PDT)
Subject: Webtools:: vs Bugzilla:: packages
In-Reply-To: <3D609358.5030808@mozilla.org>
Message-ID: <Pine.LNX.4.43.0208190100150.1843-100000@moby.sigkill.com>

On Mon, 19 Aug 2002, Gervase Markham wrote:

> Bradley Baetz wrote:
> > IOW, its a great idea in theiry, but in practice theres no point.
>
> I agree. And in the unlikely case of Bonsai using it, it wouldn't be a
> disaster if it used it out of a package called "Bugzilla", either, if we
> couldn't be bothered to move it.

Put my down for agreement... for exactly the reasons that you and Gerv
point out.

Later,
Paul
  -----------------------------------------------------------------------
  J. Paul Reed                 preed at sigkill.com || web.sigkill.com/preed
  Wait, stop!  We can outsmart those dolphins.  Don't forget: we invented
  computers, leg warmers, bendy straws, peel-and-eat shrimp, the glory
  hole, *and* the pudding cup!  -- Homer Simpson, Tree House of Horror XI



From gerv at mozilla.org  Mon Aug 19 20:36:02 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Mon, 19 Aug 2002 21:36:02 +0100
Subject: Using DBI methods
References: <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home>
Message-ID: <3D6156B2.909@mozilla.org>

> 1) Can use stuff like:
>   my ($a, $b) = $dbh->selectrow_array("SELECT a,b FROM foo WHERE 
>                                        prim_key=7");
> 
> or selectall_arrayref, and so on
> 
> which is nicer/cleaner/faster/easier to read than our sendsql+fetch stuff

I'm not convinced - I find the abstraction we have, with its one 
standard method of doing things, easy to understand and very readable. 
The syntax above is ugly :-)

> 2) We get to use DBI's error handling + exceptions - currently we only 
> check for errors in one place, which is bad.

Why? I'd say it was good - I'd much rather the error handling was 
localised than scattered throughout the code next to every database access.

> 4) No more state stack
>   - don't have to worry about forgetting to push/pop the state stack -
> simply get a new statement handle when you need one from teh globally 
> available dbh when needed.

This is hardly a burden - usually, it's really obvious when you've 
forgotten.

I've always thought our simple DB API was one of the good things about 
Bugzilla's code :-)

Gerv



From gerv at mozilla.org  Mon Aug 19 20:41:26 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Mon, 19 Aug 2002 21:41:26 +0100
Subject: "rename integrity" on the activity table?
References: <Pine.LNX.4.44.0208181225240.1283-100000@tomato.home> <3D5F261B.3090201@peshkin.net>
Message-ID: <3D6157F6.70708@mozilla.org>

Joel Peshkin wrote:
> So, I 
> implemented an update in checksetup.pl that converts the existing 
> bugs_activity records to match the new schema and then deletes the 
> unused and absent field from fielddefs.  This is fairly easy.
> 
> However, this starts an interesting precedent (which would be very 
> important to anyone using the activity log to generate reports) where we 
> would expect the activity log to accurately reflect the activity even 
> across more structural changes.

I would expect the log to tell the user what happened - exactly how it 
stores the information is not relevant. If it's easiest for us to change 
  the table format and store it differently, let's do that. I.e. do what 
you said you were doing above.

Gerv



From preed at sigkill.com  Mon Aug 19 21:33:23 2002
From: preed at sigkill.com (J. Paul Reed)
Date: Mon, 19 Aug 2002 14:33:23 -0700 (PDT)
Subject: Using DBI methods
In-Reply-To: <3D6156B2.909@mozilla.org>
Message-ID: <Pine.LNX.4.43.0208191427370.1843-100000@moby.sigkill.com>

On Mon, 19 Aug 2002, Gervase Markham wrote:

> I'm not convinced - I find the abstraction we have, with its one standard
> method of doing things, easy to understand and very readable.  The syntax
> above is ugly :-)

It may be ugly, but *every* programmer who's *ever* done anything with the
Perl DBI will recognize it immediately, and they won't have to learn
Bugzilla's odd way of doing things.

I personally like the idea that we move towards a more standardized (and
dare I say "normal") way of doing things.

> > 4) No more state stack
> >   - don't have to worry about forgetting to push/pop the state stack -
> > simply get a new statement handle when you need one from teh globally
> > available dbh when needed.
>
> This is hardly a burden - usually, it's really obvious when you've
> forgotten.

In terms of a learning curve, it is.

Bugzilla DB API is, incidentally, one of the first things I looked at when
I got involved with BZ and said to myself "Geez... why didn't they just use
the Perl DBI straight up."

I'm sure I'm not the only one who's asks that.

Out of curiosity, what *was* the reason?

Later,
Paul
  -----------------------------------------------------------------------
  J. Paul Reed                 preed at sigkill.com || web.sigkill.com/preed
  Wait, stop!  We can outsmart those dolphins.  Don't forget: we invented
  computers, leg warmers, bendy straws, peel-and-eat shrimp, the glory
  hole, *and* the pudding cup!  -- Homer Simpson, Tree House of Horror XI



From gerv at mozilla.org  Mon Aug 19 20:57:58 2002
From: gerv at mozilla.org (Gervase Markham)
Date: Mon, 19 Aug 2002 21:57:58 +0100
Subject: Website directory structure
Message-ID: <3D615BD6.3050302@mozilla.org>

At the moment, with the exception of /releases and /security, which I 
did at 2.16-time, the website has most things dumped in the root 
directory. I'd like to fix this, and make e.g. a /developer one for the 
Developers' Guide/Localisers' Guide etc., and maybe another one or two 
as well.

If anyone particularly cares how this is done, let me know - otherwise 
I'll just do what seems sensible.

Gerv



From bbaetz at student.usyd.edu.au  Mon Aug 19 22:26:22 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 20 Aug 2002 08:26:22 +1000 (EST)
Subject: Using DBI methods
In-Reply-To: <3D6156B2.909@mozilla.org>
Message-ID: <Pine.LNX.4.44.0208200823030.1461-100000@tomato.home>

On Mon, 19 Aug 2002, Gervase Markham wrote:

> > 1) Can use stuff like:
> >   my ($a, $b) = $dbh->selectrow_array("SELECT a,b FROM foo WHERE 
> >                                        prim_key=7");
> 
> I'm not convinced - I find the abstraction we have, with its one 
> standard method of doing things, easy to understand and very readable. 
> The syntax above is ugly :-)

Well, its different. We can also do stuff like have C code read the values 
into a hash key;'d off some value, rather than a separate loop in the perl

> 
> > 2) We get to use DBI's error handling + exceptions - currently we only 
> > check for errors in one place, which is bad.
> 
> Why? I'd say it was good - I'd much rather the error handling was 
> localised than scattered throughout the code next to every database access.
> 

It may be localised, but its wrong, because we aren't testing all teh
funtions - we should be using exceptions (via RaisError), and the DBI
HandleError attribute. Since a database error is always fatal for us,
HandleError can just truncate the message + die.

> > 4) No more state stack
> >   - don't have to worry about forgetting to push/pop the state stack -
> > simply get a new statement handle when you need one from teh globally 
> > available dbh when needed.
> 
> This is hardly a burden - usually, it's really obvious when you've 
> forgotten.

Well, not necessarily.

> 
> I've always thought our simple DB API was one of the good things about 
> Bugzilla's code :-)

It simple, but it doesn't allow us to use as much

Bradley



From bbaetz at student.usyd.edu.au  Mon Aug 19 22:27:34 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 20 Aug 2002 08:27:34 +1000 (EST)
Subject: Using DBI methods
In-Reply-To: <Pine.LNX.4.43.0208191427370.1843-100000@moby.sigkill.com>
Message-ID: <Pine.LNX.4.44.0208200826410.1461-100000@tomato.home>

On Mon, 19 Aug 2002, J. Paul Reed wrote:

> Out of curiosity, what *was* the reason?
> 

Looking at the old DBI version we now require, it probably wasn't possible 
to do subclassing for stuff like the shadow log to work.

Thats just a guess, though.

Bradley



From bbaetz at student.usyd.edu.au  Mon Aug 19 22:57:38 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 20 Aug 2002 08:57:38 +1000 (EST)
Subject: Webtools:: vs Bugzilla:: packages (fwd)
Message-ID: <Pine.LNX.4.44.0208200855060.1977-100000@tomato.home>

Timeless still hasn't got his email working to send to developers

Anyway, its in as Bugzilla::, and we can always move it later.

Bradley

---------- Forwarded message ----------
Date: Mon, 19 Aug 2002 13:06:59 -0400 (EDT)
From: timeless at viper.haque.net
To: Bradley Baetz <bbaetz at student.usyd.edu.au>
Cc: timeless at mac.com
Subject: Re: Webtools:: vs Bugzilla:: packages

On Mon, 19 Aug 2002, Bradley Baetz wrote:
> I think that we can revisit this if, at some point in the future, someone
> feels like maintainting bonsai.

err, no fair, baloo and i are working on bonsai. i'd really like the 
shared module. (please bounce to list)



From bugreport at peshkin.net  Tue Aug 20 00:21:15 2002
From: bugreport at peshkin.net (Joel Peshkin)
Date: Mon, 19 Aug 2002 17:21:15 -0700
Subject: "rename integrity" on the activity table?
References: <Pine.LNX.4.44.0208181225240.1283-100000@tomato.home> <3D5F261B.3090201@peshkin.net> <3D6157F6.70708@mozilla.org>
Message-ID: <3D618B7B.40109@peshkin.net>

Gervase Markham wrote:

> Joel Peshkin wrote:
>
>> So, I implemented an update in checksetup.pl that converts the 
>> existing bugs_activity records to match the new schema and then 
>> deletes the unused and absent field from fielddefs.  This is fairly 
>> easy.
>>
>> However, this starts an interesting precedent (which would be very 
>> important to anyone using the activity log to generate reports) where 
>> we would expect the activity log to accurately reflect the activity 
>> even across more structural changes.
>
>
> I would expect the log to tell the user what happened - exactly how it 
> stores the information is not relevant. If it's easiest for us to 
> change  the table format and store it differently, let's do that. I.e. 
> do what you said you were doing above.
>
> Gerv
>
> -



Ok... so Here's the proposal.....

Storing the groupset bitmask number in the activity log instead of 
listing the group names is inconsistent with the way everything else 
works and it makes no sense at all after the groupset is replaced by a 
map anyway, so the history of
groupsets should be converted to eliminate the groupet numbers at the 
same time as the groupset is eliminated from the DB.

This will make group changes look just like changes to dependencies or 
CC lists, with a list of items added or removed and queries and buglists 
referring to old historical changes will treate it just like dependency 
changes. (including bug 163362)

Doing this catches groups up to being just as good and just as much of a 
mess as other history items.

At this point, the history is accurate unless something changes name. 
 Any group change made prior to a name change for that same group that 
is already in a DB is already lost/confused information.  Any group 
change made already is in jeopardy of being lost until activity logs get 
integrity just as is true of product, asignee, cclist, and component 
changes now.

To solve this, we would need to roll through the history whenever doing 
a rename.  The simplest case of this is in cases of a product or group 
rename (ignoring deletion and resurrection for now)

UPDATE bugs_activity SET removed = 'newname' WHERE removed = 'oldname' 
AND field = getfielded('product')
UPDATE bugs_activity SET added = 'newname' WHERE added = 'oldname' AND 
field = getfielded('product')

When a product or group is deleted, we may want to prevent it from being 
changed further...  That's a bit strange, but we could do something like....
UPDATE bugs_activity SET removed = 'DEFUNCT(name)' WHERE removed = 
'name' AND field = getfielded('product')
UPDATE bugs_activity SET added = 'DEFUNCT(name)' WHERE added = 'name' 
AND field = getfielded('product')

So far, so good (we can debate how to mark the defunct items later)... 
This should work for email addresses, products, groups, platforms, 
priorities, etc...

For components and versions, this is a bit trickier...  say we have a 
component "foo" of product "a" and a component "foo" of product "b" and 
we rename a/foo to a/bar while leaving b/foo alone.....

We need to
UPDATE bugs_activity SET removed ='bar' WHERE removed='foo' AND field = 
getfieldid('component') AND  **this**
(ditto for added)

Where **this** means that either the next change to product AFTER this 
entry removed "a" OR (there is no later change to product AND the bug is 
still in product "a")

This is a bit hairy, but only has to be written once and executed rarely.  

If we do this properly,  then the acitvity logs will start to be 
accurate even through renames and schema changes.  This is vital if we 
want to do good reporting.


-Joel




From justdave at syndicomm.com  Tue Aug 20 00:35:02 2002
From: justdave at syndicomm.com (David Miller)
Date: Mon, 19 Aug 2002 20:35:02 -0400
Subject: Using DBI methods
In-Reply-To: <Pine.LNX.4.43.0208191427370.1843-100000@moby.sigkill.com>
References: <Pine.LNX.4.43.0208191427370.1843-100000@moby.sigkill.com>
Message-ID: <p05111a02b9873ecd2e08@[192.168.1.203]>

On 8/19/02 2:33 PM -0700, J. Paul Reed wrote:

> Bugzilla DB API is, incidentally, one of the first things I looked at when
> I got involved with BZ and said to myself "Geez... why didn't they just use
> the Perl DBI straight up."

I'd never seen DBI before when I started working with Bugzilla.  I thought
SendSQL() was how you did it.  Then when I started doing DBI stuff it was a
learning curve.  Now I like DBI better because you can use ? placeholders
for the data in your query and let DBI worry about escaping all your
variables for you instead of having to remember to SqlQuote everything or
not.

> I'm sure I'm not the only one who's asks that.
>
> Out of curiosity, what *was* the reason?

I think it was because Bugzilla was ported from Tcl, and Tcl didn't have DBI.
-- 
Dave Miller      Project Leader, Bugzilla Bug Tracking System
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.bugzilla.org/


From justdave at syndicomm.com  Tue Aug 20 00:37:28 2002
From: justdave at syndicomm.com (David Miller)
Date: Mon, 19 Aug 2002 20:37:28 -0400
Subject: Website directory structure
In-Reply-To: <3D615BD6.3050302@mozilla.org>
References: <3D615BD6.3050302@mozilla.org>
Message-ID: <p05111a03b9873f494ae4@[192.168.1.203]>

On 8/19/02 9:57 PM +0100, Gervase Markham wrote:

> At the moment, with the exception of /releases and /security, which I
> did at 2.16-time, the website has most things dumped in the root
> directory. I'd like to fix this, and make e.g. a /developer one for the
> Developers' Guide/Localisers' Guide etc., and maybe another one or two
> as well.
>
> If anyone particularly cares how this is done, let me know - otherwise
> I'll just do what seems sensible.

Matty and I have been thinking along these lines, too.  Here's the list
Matty sent me not too long back in IRC...

[18:54:23] <MattyT> What is Bugzilla?
[18:54:25] <MattyT> Bugzilla News
[18:54:25] <MattyT> Administrators
[18:54:25] <MattyT>   - Administrator Documentation
[18:54:25] <MattyT>   - Bugzilla Consulting
[18:54:25] <MattyT>   - Master Plan
[18:54:28] <MattyT>   - Admin Query Cookbook
[18:54:31] <MattyT>   - Reporting Bugs
[18:54:31] <MattyT>   - Security Advisories
[18:54:33] <MattyT>   - Download
[18:54:35] <MattyT>   - Status Updates
[18:54:41] <MattyT>   - Discussion
[18:54:42] <MattyT>   - Who's In Charge Here?
[18:54:44] <MattyT>   - Who Uses Bugzilla?
[18:54:46] <MattyT> Developers
[18:54:48] <MattyT>   - How To Help
[18:54:55] <MattyT>   - Master Plan
[18:54:57] <MattyT>   - Developers' Guide
[18:54:59] <MattyT>   - Reviewers' Guide
[18:55:01] <MattyT>   - Release Noting Guide
[18:55:03] <MattyT>   - Reporting Bugs
[18:55:06] <MattyT>   - Status Updates
[18:55:10] <MattyT>   - Discussion
[18:55:12] <MattyT>   - Who's In Charge Here?

Directory structure within the website could easily be organized along that
line, too.
-- 
Dave Miller      Project Leader, Bugzilla Bug Tracking System
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.bugzilla.org/


From matty at chariot.net.au  Tue Aug 20 04:05:45 2002
From: matty at chariot.net.au (MattyT)
Date: Tue, 20 Aug 2002 13:35:45 +0930
Subject: Using DBI methods
References: <Pine.LNX.4.44.0208182152550.4746-100000@tomato.home>
Message-ID: <3D61C019.5F1105CA@chariot.net.au>

Bradley Baetz wrote:

> b) In theory, Param("shutdownhtml") will currently work even for scripts
> which don't |require 'CGI.pl'|.
> 
> Pre-templatisation/2.16, the check was in PutHeader, which was too
> late (and not called for non-CGIs anyway) Now, its in CGI.pl's toplevel
> stuff.

We should probably have a standard include for non-CGIs, as well as for
CGIs.

> If you object to the removal of any of the above please let me know ASAP.

Would it be possible to do transparent SQL emulation by overriding DBI
objects, ie could you deal with things like transactions, REPLACE INTO,
subselects, etc, in a transparent way?  We could definitely do these
things explicitly, by providing subs for them as we were intending, but
transparent would be nice.

-- 
         Matthew Tuck: Software Developer & All-Round Nice Guy        
 My Short Autobiography: 1985 Grade Bin Monitor 1990 Class Clown Award
1992 Awarded Most Likely To Spontaneously Combust 1996 Crowned Galactic
         Emperor 1998 Released From Smith Psychiatric Hospital


From bbaetz at student.usyd.edu.au  Tue Aug 20 04:20:26 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 20 Aug 2002 14:20:26 +1000 (EST)
Subject: Using DBI methods
In-Reply-To: <3D61C019.5F1105CA@chariot.net.au>
Message-ID: <Pine.LNX.4.44.0208201417380.1366-100000@tomato.home>

On Tue, 20 Aug 2002, MattyT wrote:

> Bradley Baetz wrote:
> 
> We should probably have a standard include for non-CGIs, as well as for
> CGIs.
>

Right, but thats a separate issue which I don't want to deal with now
 
> > If you object to the removal of any of the above please let me know ASAP.
> 
> Would it be possible to do transparent SQL emulation by overriding DBI
> objects, ie could you deal with things like transactions, REPLACE INTO,
> subselects, etc, in a transparent way?  We could definitely do these
> things explicitly, by providing subs for them as we were intending, but
> transparent would be nice.
> 

No, the DBI doesn't have any functionality to work arround database 
servers which can't handle subselects and other basic SQL features.

Stuff like transaction support is handled by DBI, but it has to be
explicit because the programmer has to tell it when to being/end the 
transaction...

Bradley



From matty at chariot.net.au  Tue Aug 20 04:15:00 2002
From: matty at chariot.net.au (MattyT)
Date: Tue, 20 Aug 2002 13:45:00 +0930
Subject: Website directory structure
References: <3D615BD6.3050302@mozilla.org> <p05111a03b9873f494ae4@[192.168.1.203]>
Message-ID: <3D61C244.C4F346D9@chariot.net.au>

David Miller wrote:

> Matty and I have been thinking along these lines, too.  Here's the list
> Matty sent me not too long back in IRC...
>
> ...
> 
> Directory structure within the website could easily be organized along that
> line, too.

While I'm sure my proposal was along the right lines, I have always been
a little troubled about the fact that some documents appear in both
categories.

The main reason it troubled me, is that if you wanted to have a links
pane for each of these pages, the links pane should indicate which
category you're in, but that depends on where you came from.

Directory structure would be another reason this is a problem.

I'm not entirely sure what to do, but a couple of options come to mind:

- if it's in developers and administrators, just put in in
administrators
- create a third section for shared stuff
- duplicate the shared pages for each section, but with different links
panes
- use a database backend, remove our mozilla.org subwebsite, generate
pages accordingly
- use some sort of HTML preprocessing system where we can generate the
pages twice, but only write the content once

None of the above make me particularly happy unfortunately, but I don't
think there is a good solution.

-- 
         Matthew Tuck: Software Developer & All-Round Nice Guy        
 My Short Autobiography: 1985 Grade Bin Monitor 1990 Class Clown Award
1992 Awarded Most Likely To Spontaneously Combust 1996 Crowned Galactic
         Emperor 1998 Released From Smith Psychiatric Hospital


From matty at chariot.net.au  Tue Aug 20 04:22:08 2002
From: matty at chariot.net.au (MattyT)
Date: Tue, 20 Aug 2002 13:52:08 +0930
Subject: Using DBI methods
References: <Pine.LNX.4.44.0208201417380.1366-100000@tomato.home>
Message-ID: <3D61C3F0.110247B2@chariot.net.au>

Bradley Baetz wrote:

> No, the DBI doesn't have any functionality to work arround database
> servers which can't handle subselects and other basic SQL features.

I didn't ask that.  I asked whether it would be possible to extend it to
do so using OOP.

> Stuff like transaction support is handled by DBI, but it has to be
> explicit because the programmer has to tell it when to being/end the
> transaction...

I didn't ask that.  I asked if we could get it to transparently handle
the transactional SQL, by sending through or ignoring BEGIN at it's
whim.

-- 
         Matthew Tuck: Software Developer & All-Round Nice Guy        
 My Short Autobiography: 1985 Grade Bin Monitor 1990 Class Clown Award
1992 Awarded Most Likely To Spontaneously Combust 1996 Crowned Galactic
         Emperor 1998 Released From Smith Psychiatric Hospital


From bbaetz at student.usyd.edu.au  Tue Aug 20 04:48:27 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 20 Aug 2002 14:48:27 +1000 (EST)
Subject: Using DBI methods
In-Reply-To: <3D61C3F0.110247B2@chariot.net.au>
Message-ID: <Pine.LNX.4.44.0208201444220.1409-100000@tomato.home>

On Tue, 20 Aug 2002, MattyT wrote:

> Bradley Baetz wrote:
> 
> > No, the DBI doesn't have any functionality to work arround database
> > servers which can't handle subselects and other basic SQL features.
> 
> I didn't ask that.  I asked whether it would be possible to extend it to
> do so using OOP.

No. In theoyr, anything is possible. In practice, its not, since queries 
get written in diferent ways (eg you may use a differnet number of 
queries, stroe stuff in an intermediate data structure, use a subselect 
instead of ajoin, or as well as, etc).

At some point it may be worth specialising the Bugzilla::Search stuff 
(which is the only 

> 
> > Stuff like transaction support is handled by DBI, but it has to be
> > explicit because the programmer has to tell it when to being/end the
> > transaction...
> 
> I didn't ask that.  I asked if we could get it to transparently handle
> the transactional SQL, by sending through or ignoring BEGIN at it's
> whim.

Yes, that can be done, since beginning/ending transactions are method
calls (->begin_work) If the db doesn't support transactions, then the
driver will raise a fatal error whih our wrapper class can ignore (or 
possible our error handling code will just pass on it; that would be 
easier but I'm not sure if its possible)

Bradley



From matty at chariot.net.au  Thu Aug 22 03:16:15 2002
From: matty at chariot.net.au (MattyT)
Date: 22 Aug 2002 12:46:15 +0930
Subject: IMPORTANT: New Bugzilla Developer Discussion List
Message-ID: <1029986176.4582.3.camel@megagerbil>

There is now a new list for Bugzilla development discussion.  This list
was created in order to have a place to discuss development issues in an
open forum separate from administrator issues.

The list is developers at bugzilla.org.  More information, including the
archive of messages received so far, can be obtained at
"http://www.bugzilla.org/discussion.html".

Development discussion can continue on existing threads in the forum
they are in, but please raise new threads about Bugzilla development
discussion on the developers list.

The webtools list/group will remain as an area for administrator and
end-user discussion, as well as for all discussion about the other
webtools.  This change should not cause developers to be less likely to
help on webtools.

The developers list is also for review requests, as these often spawn
development discussion.  The existing reviewers list will remain closed,
and likely get renamed to reflect the fact that it is private (eg for
security-bug-related discussion), and should become low traffic.

-- 
         Matthew Tuck: Software Developer & All-Round Nice Guy        
 My Short Autobiography: 1985 Grade Bin Monitor 1990 Class Clown Award
1992 Awarded Most Likely To Spontaneously Combust 1996 Crowned Galactic
         Emperor 1998 Released From Smith Psychiatric Hospital




From bbaetz at student.usyd.edu.au  Sun Aug 25 10:38:46 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sun, 25 Aug 2002 20:38:46 +1000 (EST)
Subject: s/use diagnostics/use warnings/g ?
Message-ID: <Pine.LNX.4.44.0208252007580.15935-100000@tomato.home>

What do people think about:

a) removing |use diagnostics|;
b) adding |use warnings|;

|use diagnostics| takes just over 0.25 seconds to load, as measured by
|time perl -c -e 'use diagnostics'| This is because it parses perdiag.pod
at |use| time, including unescaping values found in the pod. (There is at
least a comment in the .pm about how demand loading crashed perl 5.001, so 
at least someone thought about the overhead)

0.25 seconds may not sound like much, but its 1/4 of the time to load
index.cgi, and 2.5 times the time it takes for ConnectToDatabase to
connect to localhost.

This means reopening bug 76923 which I closed on the grounds that |use
diagnostics| couldn't possible be taking 5% of show_bug (in fact, it takes
a bit over _15%_ locally, redirecting output to /dev/null). Sorry for
that, Zach.

(My tests before were testing runtime overhead - since this is at compile
time, you need to remove |use diagnostics| from _all_ .pm + .pl files
involved. For show_bug, this includes bug_form.pl, show_bug.cgi, CGI.pl,
globals.pl, *.pm, and Bugzilla/*.pm, and possibly a couple more)

There aren't that many warnings we run into, and |perldoc perldiag| can
help for unfamilar cases. It will also stop the logs being cluttered up
with detailed explinations which we don't want/need (This was why I
started looking into this originally)

The second thing is to add |use warnings| (instead of using -w). This
allows us to scope warnings, and make them fatal, too. It probably also
makes warnings get generated on win32, but that may be happening anyway.  
However, I have this vague recollection that thats a perl5.6 thing. If it
is, then we shouldn't do this just yet (I can't seem to logon to the
machine at my university which uses 5.005 to check this, though)

Thoughts?

Bradley



From bbaetz at student.usyd.edu.au  Sun Aug 25 16:13:04 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Mon, 26 Aug 2002 02:13:04 +1000 (EST)
Subject: Config code reorg/rewrite
Message-ID: <Pine.LNX.4.44.0208260211520.17236-100000@tomato.home>

I've posted a rewrite/reorg of the config code in bug 163829.

It got a bit out of hand, so the patch is rather large...

Its still a WIP, but I'd appreciate hearing what people think of the
direction.

Bradley



From justdave at syndicomm.com  Mon Aug 26 03:06:58 2002
From: justdave at syndicomm.com (David Miller)
Date: Sun, 25 Aug 2002 23:06:58 -0400
Subject: Config code reorg/rewrite
In-Reply-To: <Pine.LNX.4.44.0208260211520.17236-100000@tomato.home>
References: <Pine.LNX.4.44.0208260211520.17236-100000@tomato.home>
Message-ID: <p05111a02b98f4bae3438@[192.168.1.203]>

On 8/26/02 2:13 AM +1000, Bradley Baetz wrote:

> I've posted a rewrite/reorg of the config code in bug 163829.
>
> It got a bit out of hand, so the patch is rather large...
>
> Its still a WIP, but I'd appreciate hearing what people think of the
> direction.

Based on the description on the bug, this sounds like a Good Thing.  I
haven't had time to sit down and look at the patch yet though.
-- 
Dave Miller      Project Leader, Bugzilla Bug Tracking System
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.bugzilla.org/


From justdave at syndicomm.com  Mon Aug 26 03:08:37 2002
From: justdave at syndicomm.com (David Miller)
Date: Sun, 25 Aug 2002 23:08:37 -0400
Subject: s/use diagnostics/use warnings/g ?
In-Reply-To: <Pine.LNX.4.44.0208252007580.15935-100000@tomato.home>
References: <Pine.LNX.4.44.0208252007580.15935-100000@tomato.home>
Message-ID: <p05111a03b98f4be7418d@[192.168.1.203]>

On 8/25/02 8:38 PM +1000, Bradley Baetz wrote:

> The second thing is to add |use warnings| (instead of using -w). This
> allows us to scope warnings, and make them fatal, too. It probably also
> makes warnings get generated on win32, but that may be happening anyway.
> However, I have this vague recollection that thats a perl5.6 thing. If it
> is, then we shouldn't do this just yet (I can't seem to logon to the
> machine at my university which uses 5.005 to check this, though)
>
> Thoughts?

I like that idea.  More likely to get the important part of the error with
a tail /var/log/httpd/error_log that way (instead of getting the tail end
of the diagnostic message).
-- 
Dave Miller      Project Leader, Bugzilla Bug Tracking System
Lead Software Engineer/System Administrator, Syndicomm Online
http://www.syndicomm.com/            http://www.bugzilla.org/


From bbaetz at student.usyd.edu.au  Mon Aug 26 03:57:48 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Mon, 26 Aug 2002 13:57:48 +1000 (EST)
Subject: s/use diagnostics/use warnings/g ?
In-Reply-To: <Pine.LNX.4.44.0208252007580.15935-100000@tomato.home>
Message-ID: <Pine.LNX.4.44.0208261344550.10489-100000@tomato.home>

Just to put startup timings into perspective:

File::Spec takes 0.06 s
File::Temp takes 0.17 (including File::Spec time)
Template takes 0.25 s (which includes File::Temp time)
DBI takes 0.14 s

(These are 'warm' timings; I ignored the first run when the disk file 
wasn't in cache)

You can't add these together to get total time; for example both Template 
and DBI use XSLoader, but the startup cost is only paid once.

For 'fun', I note that the 'startup' time for index.cgi (ie time taken if
I add |exit 0;| just before the connecttodatabase call) is 0.87 seconds.
The total time to run the entire script > /dev/null (including db
connection, but no user auth stuff) is 0.9 seconds

Figures for show_bug are 0.9 init vs 1.2 total on a bug with 82 2-3 line 
comments.

(These figures do include |use diagnostics;|)

Bring on mod_perl! :P

Bradley



From freitag at suse.de  Thu Aug 22 13:16:17 2002
From: freitag at suse.de (Klaas Freitag)
Date: Thu, 22 Aug 2002 15:16:17 +0200 (CEST)
Subject: Thoughts about bug email
Message-ID: <Pine.LNX.4.44.0208221446110.30531-100000@subbotin.suse.de>


Hi,

I saw that it is on the TODO to incoorperate my old bug_email.pl script better
in Bugzilla. Since we here at SuSE and some other sites seem to use it,
I am currently trying to rewrite it to meet the needs of the new bugzilla.
The first step is to replace the functionality of the existing bug_email.pl
(adding bugs and comments to bugzilla) and to fix some bugs of it. In the
future, I like to make it possible even to switch stati by the tool.

Here are some thoughts about and I kindly ask for comments.

The new bug_email.pl should not only be a email script but more general
a commandline import script. It takes a bug in xml format following the
bugzilla.dtd and inserts it into a bugzilla.

To import a bug from an email needs only a conversion from the mime format to
the xml format, which is piped into the createBug script.

In order to be as independent from the bugzilla server as possible, I try to
avoid that the command line create bug script needs
a.) access to the bugzilla code base like CGI.pl etc.
b.) a connection to bugzillas database
c.) other access to the bugzilla server than over http.

That can be achieved by submitting bugs by  a scripted user agent calling
post_bug.cgi on the bugzilla server and transmitting the neccessary parameters.
The post_bug.cgi script inserts the bug to the database and notifies the
assignee (what was not done by bug_email.pl) and does all the stuff now
and in future needed.

One problem still remains: Checks for valid versions, products etc. that
comes in the xml file. The solution can be to have a script sendVersionTable.cgi
on the server, that sends the data/versioncache file to the client. The
client than saves it to a file and 'requires' it in the perl code as
usually. With the contents of the file, a lot of checks are possible,
especially if a variable %defaults exists, which holds the default values for
some bug attributes.

The described functionality works already with my test code here, which is
very alpha and needs error checking etc. But if somebody is interested, I like
to send it over.

The next step for me is to analyse the output of post_bug.cgi to check if the
action was successfull. It is not very comfortable to parse the 'human
readable' html code returned for default. Is it possible with the templates
to return a easy parseable output like xml depending on a flag coming with
the request indicating that the bug create request comes from a script?

Thanks for comments,

Klaas


-- 
  Wenn du einen Schneck behauchst                          Klaas Freitag
  Schrumpft er ins Gehaeuse,               *        mail freitag at suse.de
  Wenn du ihn in Kognak tauchst,                    SuSE Labs, Nuernberg
  Sieht er wei?e Maeuse - Ringelnatz, Ueberall                         #




From myk at mozilla.org  Mon Aug 26 17:51:35 2002
From: myk at mozilla.org (Myk Melez)
Date: Mon, 26 Aug 2002 10:51:35 -0700
Subject: s/use diagnostics/use warnings/g ?
References: <Pine.LNX.4.44.0208252007580.15935-100000@tomato.home>
Message-ID: <3D6A6AA7.7030000@mozilla.org>

Bradley Baetz wrote:

>What do people think about:
>
>a) removing |use diagnostics|;
>b) adding |use warnings|;
>  
>
Sounds like a great idea.  It would help our users much more than it 
would hurt our developers.  Let's do it.

-myk




From bbaetz at student.usyd.edu.au  Mon Aug 26 21:59:42 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Tue, 27 Aug 2002 07:59:42 +1000 (EST)
Subject: s/use diagnostics/use warnings/g ?
In-Reply-To: <3D6A6AA7.7030000@mozilla.org>
Message-ID: <Pine.LNX.4.44.0208270759160.1358-100000@tomato.home>

On Mon, 26 Aug 2002, Myk Melez wrote:

> Bradley Baetz wrote:
> 
> >What do people think about:
> >
> >a) removing |use diagnostics|;
> >b) adding |use warnings|;
> >  
> >
> Sounds like a great idea.  It would help our users much more than it 
> would hurt our developers.  Let's do it.

I did a) yesterday. :)

use warnings is 5.6 only, so thats out for the moment.

Bradley



From kiko at async.com.br  Fri Aug 30 00:17:58 2002
From: kiko at async.com.br (Christian Reis)
Date: Thu, 29 Aug 2002 21:17:58 -0300
Subject: Thoughts about bug email
In-Reply-To: <Pine.LNX.4.44.0208221446110.30531-100000@subbotin.suse.de>; from freitag@suse.de on Thu, Aug 22, 2002 at 03:16:17PM +0200
References: <Pine.LNX.4.44.0208221446110.30531-100000@subbotin.suse.de>
Message-ID: <20020829211758.N7981@blackjesus.async.com.br>

On Thu, Aug 22, 2002 at 03:16:17PM +0200, Klaas Freitag wrote:
> 
> Here are some thoughts about and I kindly ask for comments.
> 
> The new bug_email.pl should not only be a email script but more general
> a commandline import script. It takes a bug in xml format following the
> bugzilla.dtd and inserts it into a bugzilla.
>
> To import a bug from an email needs only a conversion from the mime format to
> the xml format, which is piped into the createBug script.

It would still need to parse the email, at any rate, so I guess having
two files that worked together (probably a library) may be the best
approach. You'd end up with an XML parsing set of functions that did the
database operations, and the mail parser that took care of banging the
text into a decent format.

> In order to be as independent from the bugzilla server as possible, I try to
> avoid that the command line create bug script needs
> a.) access to the bugzilla code base like CGI.pl etc.
> b.) a connection to bugzillas database
> c.) other access to the bugzilla server than over http.

Ahm, I start to understand your idea. But why is it important to have this
separate from the bugzilla server? Or is that just one of the design
goals? Do you think it will be a common case to have the mail handling
separate from the bugzilla server?

If it is deemed very important, I'd agree with moving to an HTTP
interface between mailparse and post_bug.cgi (if that's the current
plan, as I understand it).

> One problem still remains: Checks for valid versions, products etc. that
> comes in the xml file. The solution can be to have a script sendVersionTable.cgi
> on the server, that sends the data/versioncache file to the client. The
> client than saves it to a file and 'requires' it in the perl code as
> usually. With the contents of the file, a lot of checks are possible,
> especially if a variable %defaults exists, which holds the default values for
> some bug attributes.

I'm not sure enough to comment on this. Keep in mind there are security
issues involved with this: would this not work for private products and
bugs?

> The described functionality works already with my test code here, which is
> very alpha and needs error checking etc. But if somebody is interested, I like
> to send it over.

Would be nice to see this attached to the relevant bug :-)

> The next step for me is to analyse the output of post_bug.cgi to check if the
> action was successfull. It is not very comfortable to parse the 'human
> readable' html code returned for default. Is it possible with the templates

We do that in mozbot for show_bug.cgi, but we may want to offer a
simpler API for RPC (since that's what we are doing here, I guess)

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL


From preed at sigkill.com  Fri Aug 30 17:02:03 2002
From: preed at sigkill.com (J. Paul Reed)
Date: Fri, 30 Aug 2002 10:02:03 -0700 (PDT)
Subject: Email Notification Reviews
Message-ID: <Pine.LNX.4.43.0208300944260.12022-100000@moby.sigkill.com>


Devs:

I'll shortly be returning to school when my available free time will tend
towards nil.

Therefore, before I go back, I'd like to get the following bugs checked in,
but I need YOUR help to do it (imagine a poster of JayPee with top hat
pointed at you):

Bug 84876: The infamous sendmail cleanup patch. Both Gerv and I have
           submitted our respective patches; both are what I would call
           'proof-of-concept' patches, since after we decide on which one
           to use, both will still need some work.

Our last set of patches bitrotted for 4 months; we need to come to a final
decision on this and get it checked in.

I think the best way to do this is to get all available reviews/comments on
these two different approaches, and if that doesn't solve the problem, then
justdave can give us some final feedback and we can get going on this.

Anyway, 84876 just needs to be finished... and considering there are two
patches waiting to fix this bug, we have no excuses now.

Bug 124174: Processmail as a package. I'm driving this to checkin for
            bbaetz. The v5 patch was never reviewed. Also, Jouni: could
            you test on Win32?

These are the two biggies right now, and if I can get those checked in, I
can start working on other bugs like processmail locking (133368) and some
of the many enhancements filed against the Email Notifications product.

reviewers'@ help would be greatly appreciated.

Thanks gang!

Later,
Paul
  -----------------------------------------------------------------------
  J. Paul Reed                 preed at sigkill.com || web.sigkill.com/preed
  Wait, stop!  We can outsmart those dolphins.  Don't forget: we invented
  computers, leg warmers, bendy straws, peel-and-eat shrimp, the glory
  hole, *and* the pudding cup!  -- Homer Simpson, Tree House of Horror XI



From bbaetz at student.usyd.edu.au  Sat Aug 31 04:41:05 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sat, 31 Aug 2002 14:41:05 +1000 (EST)
Subject: Exception modules?
Message-ID: <Pine.LNX.4.44.0208311436180.6645-100000@tomato.home>

Anyone know a good exception-style module for perl, which we can use in 
bugzilla?

As more stuff moves to modules which may be resued from non-CGI scripts,
we really should have a way of reporting errors apart from using die.

There are a few on CPAN, none of which look like they've been updated in a 
while, though.

(Error.pm, Exception.pm are the main ones)

http://axkit.org/docs/presentations/tpc2001/exceptions.axp/a.pdf is 
interesting, too.

Bradley



From bbaetz at student.usyd.edu.au  Sat Aug 31 04:58:29 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sat, 31 Aug 2002 14:58:29 +1000 (EST)
Subject: Exception modules?
In-Reply-To: <Pine.LNX.4.44.0208311436180.6645-100000@tomato.home>
Message-ID: <Pine.LNX.4.44.0208311446240.6778-100000@tomato.home>

On Sat, 31 Aug 2002, Bradley Baetz wrote:

> (Error.pm, Exception.pm are the main ones)
> 

... and Exception::Class/Exception::Class::DBI are actually maintainted 
(OK, ::DBI was started last week, but theresnot that much we want, anyway.

They do bring in a fair number of dependencies, mainly to handle stack 
traces.

As well, Exception::Class has stack trace problems with perl 5.6.0 (not
5.6.1), because of bugs in perl. I thikn we can live with that, though

ThrowCodeError/ThrowUserError would then throw real errors whose default 
behaviour would be to call into the template. Or something along those 
lines.

Bradley



From jth at mikrobitti.fi  Sat Aug 31 09:10:02 2002
From: jth at mikrobitti.fi (Jouni Heikniemi)
Date: Sat, 31 Aug 2002 12:10:02 +0300 (EEST)
Subject: Email Notification Reviews
In-Reply-To: <Pine.LNX.4.43.0208300944260.12022-100000@moby.sigkill.com>
Message-ID: <Pine.NEB.4.21.0208311206280.13989-100000@dns.mikrobitti.fi>

On Fri, 30 Aug 2002, J. Paul Reed wrote:

>             bbaetz. The v5 patch was never reviewed. Also, Jouni: could
>             you test on Win32?

I plan to, really soon now. Sorry it has taken so long. :-(

If somebody else has the time to test that one soonish, I'd love to hear
the results. Converting that command-line interface may have some problems
wrt passing the force* lists. While v1 of the patch worked fine,
processmail internals have had lots of changes since then (at least Myk's
unco mail pref patch rewrote major parts of the logic). I hope that won't
be a problem, though.


Jouni



From bbaetz at student.usyd.edu.au  Sat Aug 31 09:28:48 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sat, 31 Aug 2002 19:28:48 +1000 (EST)
Subject: Who uses data/sqllog?
Message-ID: <Pine.LNX.4.44.0208311840540.2052-100000@tomato.home>

Emulating data/sqllog exactly is going to be a big pain. So, I'd like to 
know who actually uses it, and if they do, what for?

I believe that the original purpose was to log slow queries, but mysql now 
handles that with its own option.

I presume that bmo doesn't use it, because the exclusive locking would 
probably slow stuff down. It doesn't work on windows < winnt, 
either, according to perlport.pod.

DBI does have a Trace variable which can be used, but its a lot more 
verbose, even on its lowest setting;

use Bugzilla::DBI;
my $dbh = Bugzilla::DBI->connect();
print $dbh->selectrow_array("SELECT bug_id FROM bugs WHERE bug_id = 4");
print $dbh->selectrow_array("SELECT bug_id FROM bugs WHERE bug_id = 3");

gives:

    DBI 1.30-nothread dispatch trace level set to 1
1   <- prepare('SELECT bug_id FROM bugs WHERE bug_id = 4')= 
Bugzilla::DBI::st=HASH(0x832fb08) at DBI.pm line 111
    <- FETCH('Statement')= 'SELECT bug_id FROM bugs WHERE bug_id = 4' 
('Statement' from cache) at - line 3
    <- selectrow_array('SELECT bug_id FROM bugs WHERE bug_id = 4')= ( '4' 
) [1 items] at - line 3
    <- DESTROY= undef at - line 4
1   <- prepare('SELECT bug_id FROM bugs WHERE bug_id = 3')= 
Bugzilla::DBI::st=HASH(0x8348800) at DBI.pm line 111
    <- FETCH('Statement')= 'SELECT bug_id FROM bugs WHERE bug_id = 3' 
('Statement' from cache) at - line 4
    <- selectrow_array('SELECT bug_id FROM bugs WHERE bug_id = 3')= ( '3' 
) [1 items] at - line 4
    <- DESTROY= undef
    <- disconnect_all= '' at DBI.pm line 565
    <- DESTROY= undef during global destruction
    <- DESTROY= undef during global destruction

when tracing is turned on after connecting. It also prints the values that 
its fetching.

On higher trace levels, DBI prints more information than you'd ever want
to know.

So, the choices are:

1) If people only use this for occasional debugging, then I'll just set
DBI up to trace to data/sqllog if it exists.

2) I can keep the current implementation basically as-is

3) Something else.

Thoughts?

Bradley



From bbaetz at student.usyd.edu.au  Sat Aug 31 10:14:27 2002
From: bbaetz at student.usyd.edu.au (Bradley Baetz)
Date: Sat, 31 Aug 2002 20:14:27 +1000 (EST)
Subject: Who uses data/sqllog?
In-Reply-To: <Pine.LNX.4.44.0208311840540.2052-100000@tomato.home>
Message-ID: <Pine.LNX.4.44.0208312012170.2490-100000@tomato.home>

On Sat, 31 Aug 2002, Bradley Baetz wrote:

> Emulating data/sqllog exactly is going to be a big pain.

... actually, while it is a pain, I have to do it anyway to support our 
shadowdb stuff. So I'm still curious about the answers, but its going to 
stay anyway, at least for the time being.

Bradley