Custom Fields again

[Ludovic Dubost]

Hi,

Let me join in this discussion which seems very important to me.
First let me summarize my point of view:

- custom fields have to be implemented one way or another.

To be able to explain why I have this point of view let me share a 
little the way we have been using bugzilla internally at our company:

Today we are using bugzilla for multiple areas:

- bug tracking
- project management (tasks)
- internal help desk
- customer service
- pre-sales
- external help desk

Actually we have one separate external help desk bugzilla installation 
but all there other areas are all handled inside the same bugzilla 
installation. For these usage we have developped some additional 
features that I still need to submit to the bugzilla community (I'll 
write a separate email to see which ones would be the most interesting 
to submit first).

There are are a lot of advantages of being able to handle all these 
areas in the same installation. For example we can refer development 
bugs to projects that are refered to pre-sales request or customer 
service request. For this we are making a heavy use of dependencies 
between all the tasks (for us the term 'bug' is actually a synonym for 
the work 'task').

There is no way we could have unleashed the power of bugzilla without 
the custom fields. We have bugzilla wide custom fields and we have area 
(product in the bugzilla language) specific custom fields. For example 
in the pre-sales request and customer service area we have a 'Customer' 
field (actually  we have added a hardwired field for customer before the 
custom field code was available). In one of the project management areas 
specifically dedicated for production (we produce reports) we have 
custom fields detailling what the clients have bought. One the other 
hand, the OS and System field are barely used in our installation (a 
little in the help desk section) and would have been much better as 
product specific custom fields.

Now to the implementation, clearly the joined table is clearly a 
complexity issue. However it allows to create multiple entry fields 
which is interesting. I regret that there isn't a date type field. I 
believe limiting the number of custom fields per area is a little 
dangerous as in certain installation there is a risk that you need more 
that you are actually allowed.

Now what could be done to simplify all this. There are here general 
ideas that go beyond the custom field code and that are due to lots of 
hours of bugzilla hacking:

- More modularization of the bug handling code:

* Currently there is a Bug.pm object that is only used for the XML 
export. It could be use to read/save/manipulate bug in an object form. 
This object would also include all the security features that are 
repeated all over the bugzilla code.

* There is a lot of code duplicated to handle the same code. Custom 
fields uses field types. These types could be used globally and the same 
code could be called when the fields are of the same type to:

   - read the FORM data and transform it in internal data and 
insert/update/search SQL
   - write the HTML for viewing, editing or querying the field

* With this using a field or a feature requiring additional fields would 
mean only turning on the definition of the fields.

* Also if you change the fields behavior in one area of the code it will 
change it globally.

* If there is this modularization then if will become much less 
important to store the data one way or another as this will be handle by 
the field type object and not by user code. You could have the first 2 
fields per type handled by internal columns and the additional fields 
handled by a separate table. You could also have one main table for the 
main fields and a subtable with empty fields of different types for the 
extension.

* Also you could add security to specific fields (make them only 
viewable by certain groups).

* There are solutions to handle multiple entry fields in one integer 
field using a storage using binary combination (2^n + 2(n-1) + ..) and 
lookup tables between IDs and Strings. The could be transparent to the 
bugzilla code using the field type objects I've been talking about 
previously.

- A separate table with the custom fields could be an interesting 
solution. For this to work the binary field with lookup table would need 
to be used for multiple entry fields and sufficient columns of each type 
would need to be created:

* You would get only one line per bug.
* This line does not need to be created if there are no custom fields in 
the bug.
* You can query it with one join which would clearly makes things simpler.
* You don't need to read it if you don't need the custom field data.
* If really it is needed you could create a secondary line if you need 
additional custom fields (but that seems a little complicated).
* You can do storage by type (id, numbers, strings, dates).

To summarize, I think that bugzilla is way much more powerfull with 
custom fields. In terms of code it would be much more easy to program if 
the fields where abstracted inside a bug object (Bug.pm) for all 
features (currently you can use that for reading).

Ludovic

Gervase Markham wrote:
> Over in
> http://bugzilla.mozilla.org/show_bug.cgi?id=91037
> Dave said:
> 
> "The current thinking is that we decide what the core types of these 
> fields are that we need to have available, and provide 3 or 4 of each 
> hardwired in the bugs table with a UI in the admin interface for 
> customizing the labels shown to the users or hiding/showing those 
> fields.  This gives you a limited ability for adding custom fields, 
> while not creating headaches and performance issues with multiple joins 
> during queries."
> 
> This solution seems to me to get the worst features of custom fields 
> without getting any of the good ones. We all know that
> http://www.joelonsoftware.com/news/20020912.html
> is why we are considering not having them. Joel argues that having 
> custom fields leads to people defining fields they don't really need, 
> field proliferation, and a a general muddle. The above plan doesn't 
> avoid that - it merely limits admins to 3 confusing email fields, 3 
> confusing multi-selects and 3 confusing text fields.
> 
> In addition, this not-quite-generalisation means our code will be 
> scattered with references to generic_text_field1, generic_multiselect2 
> or whatever we decide to call these generic fields when we add them to 
> the bugs table. These will be carried around whether the user is using 
> them or not.
> 
> An advantage given above is that general custom fields mean performance 
> issues with multiple joins during queries, and this method avoids that. 
> But this is thinking from an engineering point of view, not a usability 
> one. If this is a problem, then big installations can content themselves 
> with the default fields and do no joins. Or they can use mod_perl to 
> improve performance in other ways. Or get a better database, that does 
> lots of joins quicker. Or we could optimise our queries in a smarter way.
> 
> Enough of this halfway house! :-) Let's have full custom fields, 
> implemented in a clean, generic and sensible way, or not at all. :-)
> 
> Gerv
> 
> ----
> To view or change your list settings, click here:
> <http://bugzilla.org/cgi-bin/mj_wwwusr?user=ludovic@netvalue.com>