Partial custom fields design document

[Steven Suson]

I've tried to stay our of this, despite my company's strong need for 
this functionality. But, I can't keep quiet anymore... After 25 yrs. of 
programming, it's continually been my experience that it is always best 
to get something into the hands of the users as soon as possible. This 
gives them the opportunity to comment, complain, etc. People like Sean 
have tried to do this, but the discussion seems to be complicating 
things more and more... further delaying ANY sort of implementation. If 
I'm out of line here, feel free to flame me, but it's my 2 cents worth.

Steven Suson

MattyT wrote:

>On Fri, 2004-03-12 at 13:10, Sean McAfee wrote:
>
>  
>
>>It's been suggested that the best way to kick-start the process of getting
>>custom fields functionality into the core is to start with a design document
>>that all can agree on.  Makes sense to me.  Accordingly, I've started
>>drawing up a document that describes the design of my latest custom fields
>>patch.
>>    
>>
>
>I didn't read this in depth, so apologies if I missed something, but I
>skimmed it, and saw what I believe to be issues I have.
>
>Meta
>----
>
>While schema and programmatic interfaces are important, the details of
>these are also uninteresting and not major.
>
>We need a document that answers the major questions of how the system
>hangs together, such as:
>
>- how are fields added and removed?  what happens to the data on
>removal?
>- how are fields configured?
>- how will custom fields interact with the templates - will custom
>fields automatically appear without hacking?
>- which existing fields will be able to become custom fields
>eventually?  if some can't, why not?
>- how does your system interact with I18N?  multi-language bugzillas and
>the existing fields are going to need some I18N support, which must
>entail some change in the way I18N works
>- what are the types of fields, and what customisation can be done to
>them?  eg if i wanted text fields to regexp restrictions, how would i
>add this functionality?  what about more general restrictions? 
>inter-field restrictions?
>- general schema details - is the table metadata stored in the db?  is
>there one or multiple multi-value fields per table?  are single-valued
>fields implemented by a bugs table field?  how will the existing fields
>be able to fit into this?
>- are you going to be able to display multi-valued enumeration fields on
>the bug list?  if so, how?
>
>Also, there needs to be a rationale for the decisions, not just
>details.  Otherwise everyone's going to be arguing about things you
>might have already figured out.
>
>Once we have all this, then we can move onto details.  After there's
>some kind of consensus between this rabble, the details will likely be
>easy in comparison.  And this is the biggest change that has ever
>happened to Bugzilla, so doubly so.
>
>So put an overall summary, then for each section, a summary, then the
>details.  Mere mortals don't have the stomach to deal with all this
>implementation detail at once, this will help us.
>
>Details
>-------
>
>The idea of having all these generic tables (packing multiple fields
>into one table) I find somewhat ugly.  Not only does it make a
>dev's/admin's job of analysing a single field far more complicated, it
>makes Bugzilla's job more complicated too - not only the SQL, but it
>must either index on the field or a value, I think doing both is
>difficult, thought probably possible.
>
>The question here is whether you ever want to analyse more than one
>field at once, and I'm pretty sure the answer is no for most field types
>(the only obvious exception is relationship fields (for dep trees) which
>I don't think you even defined, although they're not really fields in
>some sense, since they're between bugs).
>
>Given that you don't need this, I think each field should have its own
>bugs field or separate table as per now.  I can understand the current
>system doesn't require any schema changes, but I don't see that as a big
>enough issue to warrant doing things this way.
>
>Furthermore, I'd avoid putting "custom fields" on everything, since the
>intention of any custom fields system should be to eventually subsume
>almost all fields, making "custom fields" a rather useless description.
>
>I originally had objections to adding and removing fields from the web
>interface, but the more I think about it the less of a problem it
>seems.  It's certainly a rare enough operation that it could be done
>locally without major issues, however.
>
>What I think we definitely do need, is to have modules controlling each
>field (sub)type.  This will allow customisation through inheritance, and
>hence will benefit administrators greatly, since they can plug in field
>types.  Over time the field modules would gain more functionality and
>hence more separate-file customisation capability.
>
>We're never going to catch everything admins will want through a web
>configuration system, but I think with bug field modules, we can get
>99.9% through inheritance.
>
>This being said, the intention is of course to make this all reasonably
>easy.  Ideally you'd be able to avoid any Perl work here for simple
>customisation, but there should be generous scope for Perl
>customisation.
>
>My Proposal
>-----------
>
>There's no need to do this all immediately as one patch.  But I'd want
>to see us on this path.
>
>A bug field module is placed in Bugzilla/BugFields/Field.pm.  This
>contains all the code necessary for handling the field type, various
>events to override, defines its configuration parameters, and so on.
>
>It either directly inherits from a module for its field type (eg
>enumeration, text, date, etc), or directly inherit from another abstract
>module which inherits from this, providing some shared functionality.
>
>The field modules are only templates, and you can instantiate them
>multiple times - each time you would have a different config.
>
>The field list (name and type) would be in the database because it's a
>good idea to put the metadata with the data (eg if you copied the
>database).  I don't have any objection to having a web interface here
>because it allows us to gracefully handle field deletion with prompts,
>and asking whether to remove the old field/table, which something like
>localconfig is famous for not doing.
>
>Each field module would define its configuration parameters and you
>could then set it up on a field config page.  For a general text field,
>it might allow configuring an acceptable regex, but for a more specific
>one, it might be more appropriate hardcode the regex in Perl rather than
>configure it.  The field configuration can be stored as name/value
>strings on one table.
>
>I would modify the templates to automatically display all the fields. 
>It would know information such as the fields priority (first section,
>second section, etc), type, and so on, and could use this information to
>order the fields in a similar way they are now.
>
>This is going to shift a variety of strings out of the template.  Two
>possibilities come to mind about I18N.
>
>The first is each field must have a template fragment defining these
>strings, which can get used by whatever templates display this field.
>
>The second places the strings elsewhere.  Most of the strings would be
>in the field modules.  For general field modules, some of the strings
>would be in the field configuration.
>
>The first means that all I18N remains in one place, the templates.
>
>The second however means that a module could come complete with I18N as
>one file without needing a separate template, which is good for
>pluggability.
>
>The strings in the field configuration would need a separate I18N
>capability for a multi-language installation, but that's an
>administrator's task, not a translator's task.  This way an admin can
>set all this up from one place.
>
>I18N would be a matter of patching the module, instead of creating a
>copy of the template which could get out of sync.
>
>I would expect to see various types of fields eventually - enumeration
>(eg OS), text (eg URL), date/time, allocation (eg votes), account (eg
>reporter) and relationship (eg dependencies).
>
>I would expect to leave multi-valued custom fields off of the bug list
>for now, as we're likely to have subselects that can do this before we
>get to hardcore porting of the old fields over.
>  
>