From gerv at mozilla.org Wed Dec 4 00:27:28 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 04 Dec 2002 00:27:28 +0000 Subject: Overload... Message-ID: <3DED4BF0.3090208@mozilla.org> Guys, There are a few reviews outstanding against me; I just wanted to say that I haven't forgotten them, but am currently suffering a bit from overload and backlog. I've cleared a lot tonight, but it's now 12.20am so I'm going to bed. Tomorrow night, I hope to get back on track a bit. Gerv From myk at mozilla.org Fri Dec 6 02:25:04 2002 From: myk at mozilla.org (Myk Melez) Date: Thu, 05 Dec 2002 18:25:04 -0800 Subject: RT hackathon this/next weekend Message-ID: <3DF00A80.7020108@mozilla.org> Joel recently suggested we do a request tracker hackathon and get rid of all the niggling bugs on the RT clean-up meta-bug. http://bugzilla.mozilla.org/show_bug.cgi?id=rt-clean-up I think a hackathon is a great idea, but I'm a bit booked up this weekend. I could probably do Sunday (December 8) 9am-2pm PST (UTC-08:00). Is anyone interested in joining me and Joel? How's that time for you? Would the following weekend be better? -myk From bbaetz at student.usyd.edu.au Fri Dec 6 07:16:22 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Fri, 6 Dec 2002 18:16:22 +1100 Subject: RT hackathon this/next weekend In-Reply-To: <3DF00A80.7020108@mozilla.org> References: <3DF00A80.7020108@mozilla.org> Message-ID: <20021206071622.GA1170@mango.home> I'll be at work then - Fri your time is better for me, usualy, because thtas a weekend. That or sometime after midnight for you... Bradley From dkl at redhat.com Fri Dec 6 19:53:09 2002 From: dkl at redhat.com (Dave Lawrence) Date: 06 Dec 2002 14:53:09 -0500 Subject: Case in-sensitive email addresses Message-ID: <1039204389.10127.15.camel@mrhanky.devel.redhat.com> We have had some complaints about the way Bugzilla handles login_names in the database through channel partners as well as bug reports. Currently it seems that bugzilla accepts Dkl at Redhat.Com and dkl at redhat.com as 2 separate accounts even though they are both the same real email address. Looking at the CVS code this looks to be the same now also. Should we be lowercasing the email address when a new user is created and then lowercase values before comparing them to the login_name field such as confirm_login? Is there already a bug report for something like this? I was unable to find one at first glance in b.m.o. I think this has been discussed before but not sure what opinions came from it. Thanks From timeless at myrealbox.com Fri Dec 6 19:54:49 2002 From: timeless at myrealbox.com (timeless) Date: Fri, 06 Dec 2002 14:54:49 -0500 Subject: Case in-sensitive email addresses In-Reply-To: <1039204389.10127.15.camel@mrhanky.devel.redhat.com> References: <1039204389.10127.15.camel@mrhanky.devel.redhat.com> Message-ID: <3DF10089.5010604@myrealbox.com> Dave Lawrence wrote: > We have had some complaints about the way Bugzilla handles login_names > in the database through channel partners as well as bug reports. > Currently it seems that bugzilla accepts Dkl at Redhat.Com and > dkl at redhat.com as 2 separate accounts even though they are both the same > real email address. Looking at the CVS code this looks to be the same > now also. Should we be lowercasing the email address when a new user is > created and then lowercase values before comparing them to the > login_name field such as confirm_login? Is there already a bug report > for something like this? I was unable to find one at first glance in > b.m.o. I think this has been discussed before but not sure what opinions > came from it. um. that's a no no. email addresses are allowed to be case sensitive. From dkl at redhat.com Fri Dec 6 20:32:17 2002 From: dkl at redhat.com (Dave Lawrence) Date: 06 Dec 2002 15:32:17 -0500 Subject: Case in-sensitive email addresses In-Reply-To: <3DF10089.5010604@myrealbox.com> References: <1039204389.10127.15.camel@mrhanky.devel.redhat.com> <3DF10089.5010604@myrealbox.com> Message-ID: <1039206736.10128.36.camel@mrhanky.devel.redhat.com> When I send mail to all of my email addresses and twiddle with the case, they still seem to make it where they are supposed to go so some MTA's are munging this. 3.4.7. CASE INDEPENDENCE Except as noted, alphabetic strings may be represented in any combination of upper and lower case. The only syntactic units August 13, 1982 - 14 - RFC #822 Standard for ARPA Internet Text Messages which requires preservation of case information are: - text - qtext - dtext - ctext - quoted-pair - local-part, except "Postmaster" When matching any other syntactic unit, case is to be ignored. For example, the field-names "From", "FROM", "from", and even "FroM" are semantically equal and should all be treated ident- ically. When generating these units, any mix of upper and lower case alphabetic characters may be used. The case shown in this specification is suggested for message-creating processes. Note: The reserved local-part address unit, "Postmaster", is an exception. When the value "Postmaster" is being interpreted, it must be accepted in any mixture of case, including "POSTMASTER", and "postmaster". So from this I gather that the username portion of the email address should treated as case-sensitive and the domain part as in-sensitive. So I imagine that this should stay the way it is. Closing bugs as WONTFIX. On Fri, 2002-12-06 at 14:54, timeless wrote: > Dave Lawrence wrote: > > We have had some complaints about the way Bugzilla handles login_names > > in the database through channel partners as well as bug reports. > > Currently it seems that bugzilla accepts Dkl at Redhat.Com and > > dkl at redhat.com as 2 separate accounts even though they are both the same > > real email address. Looking at the CVS code this looks to be the same > > now also. Should we be lowercasing the email address when a new user is > > created and then lowercase values before comparing them to the > > login_name field such as confirm_login? Is there already a bug report > > for something like this? I was unable to find one at first glance in > > b.m.o. I think this has been discussed before but not sure what opinions > > came from it. > > um. that's a no no. email addresses are allowed to be case sensitive. > > ---- > To view or change your list settings, click here: > From gerv at mozilla.org Tue Dec 10 18:12:19 2002 From: gerv at mozilla.org (Gervase Markham) Date: Tue, 10 Dec 2002 18:12:19 +0000 Subject: Custom Fields again Message-ID: <3DF62E83.70707@mozilla.org> Over in http://bugzilla.mozilla.org/show_bug.cgi?id=91037 Dave said: "The current thinking is that we decide what the core types of these fields are that we need to have available, and provide 3 or 4 of each hardwired in the bugs table with a UI in the admin interface for customizing the labels shown to the users or hiding/showing those fields. This gives you a limited ability for adding custom fields, while not creating headaches and performance issues with multiple joins during queries." This solution seems to me to get the worst features of custom fields without getting any of the good ones. We all know that http://www.joelonsoftware.com/news/20020912.html is why we are considering not having them. Joel argues that having custom fields leads to people defining fields they don't really need, field proliferation, and a a general muddle. The above plan doesn't avoid that - it merely limits admins to 3 confusing email fields, 3 confusing multi-selects and 3 confusing text fields. In addition, this not-quite-generalisation means our code will be scattered with references to generic_text_field1, generic_multiselect2 or whatever we decide to call these generic fields when we add them to the bugs table. These will be carried around whether the user is using them or not. An advantage given above is that general custom fields mean performance issues with multiple joins during queries, and this method avoids that. But this is thinking from an engineering point of view, not a usability one. If this is a problem, then big installations can content themselves with the default fields and do no joins. Or they can use mod_perl to improve performance in other ways. Or get a better database, that does lots of joins quicker. Or we could optimise our queries in a smarter way. Enough of this halfway house! :-) Let's have full custom fields, implemented in a clean, generic and sensible way, or not at all. :-) Gerv From louie at ximian.com Tue Dec 10 19:00:55 2002 From: louie at ximian.com (Luis Villa) Date: 10 Dec 2002 14:00:55 -0500 Subject: Custom Fields again In-Reply-To: <3DF62E83.70707@mozilla.org> References: <3DF62E83.70707@mozilla.org> Message-ID: <1039546854.3649.236.camel@10-0-0-220.boston.ximian.com> On Tue, 2002-12-10 at 13:12, Gervase Markham wrote: > This solution seems to me to get the worst features of custom fields > without getting any of the good ones. We all know that > http://www.joelonsoftware.com/news/20020912.html > is why we are considering not having them. Joel argues that having > custom fields leads to people defining fields they don't really need, > field proliferation, and a a general muddle. The problem with Joel (as always) is that Joel knows that he is smarter than you. And Joel is /always/ smarter than you. Which may be very frequently true- certainly Joel is a smart guy, and he gives some very good examples where custom fields aren't necessary but were requested anyway, and I have several examples of my own. The problem is that I also have examples of where they are quite needed, and 2+ years of working with bugzilla doesn't seem to give me any way out short of custom fields. Joel's article irritates me because he basically denies I and my very real problems, which continue to complicate/block an upgrade to 2.16, exist. Well, they exist, but if I were as smart as him, I'd find a solution that didn't involve custom fields. [Or that's what he thinks. I tend to disagree with him. :) > Enough of this halfway house! :-) Let's have full custom fields, > implemented in a clean, generic and sensible way, or not at all. :-) Agreed that the suggested implementation seems... unpleasant. My vote, though, is Real Custom Fields, not none at all. Luis [and sadly, I still don't have any time to get this done myself, so... my vote probably doesn't/shouldn't count much except as a high-profile user. :/ From ludovic at netvalue.com Tue Dec 10 19:50:30 2002 From: ludovic at netvalue.com (Ludovic Dubost) Date: Tue, 10 Dec 2002 20:50:30 +0100 Subject: Custom Fields again In-Reply-To: <3DF62E83.70707@mozilla.org> References: <3DF62E83.70707@mozilla.org> Message-ID: <3DF64586.1030304@netvalue.com> Hi, Let me join in this discussion which seems very important to me. First let me summarize my point of view: - custom fields have to be implemented one way or another. To be able to explain why I have this point of view let me share a little the way we have been using bugzilla internally at our company: Today we are using bugzilla for multiple areas: - bug tracking - project management (tasks) - internal help desk - customer service - pre-sales - external help desk Actually we have one separate external help desk bugzilla installation but all there other areas are all handled inside the same bugzilla installation. For these usage we have developped some additional features that I still need to submit to the bugzilla community (I'll write a separate email to see which ones would be the most interesting to submit first). There are are a lot of advantages of being able to handle all these areas in the same installation. For example we can refer development bugs to projects that are refered to pre-sales request or customer service request. For this we are making a heavy use of dependencies between all the tasks (for us the term 'bug' is actually a synonym for the work 'task'). There is no way we could have unleashed the power of bugzilla without the custom fields. We have bugzilla wide custom fields and we have area (product in the bugzilla language) specific custom fields. For example in the pre-sales request and customer service area we have a 'Customer' field (actually we have added a hardwired field for customer before the custom field code was available). In one of the project management areas specifically dedicated for production (we produce reports) we have custom fields detailling what the clients have bought. One the other hand, the OS and System field are barely used in our installation (a little in the help desk section) and would have been much better as product specific custom fields. Now to the implementation, clearly the joined table is clearly a complexity issue. However it allows to create multiple entry fields which is interesting. I regret that there isn't a date type field. I believe limiting the number of custom fields per area is a little dangerous as in certain installation there is a risk that you need more that you are actually allowed. Now what could be done to simplify all this. There are here general ideas that go beyond the custom field code and that are due to lots of hours of bugzilla hacking: - More modularization of the bug handling code: * Currently there is a Bug.pm object that is only used for the XML export. It could be use to read/save/manipulate bug in an object form. This object would also include all the security features that are repeated all over the bugzilla code. * There is a lot of code duplicated to handle the same code. Custom fields uses field types. These types could be used globally and the same code could be called when the fields are of the same type to: - read the FORM data and transform it in internal data and insert/update/search SQL - write the HTML for viewing, editing or querying the field * With this using a field or a feature requiring additional fields would mean only turning on the definition of the fields. * Also if you change the fields behavior in one area of the code it will change it globally. * If there is this modularization then if will become much less important to store the data one way or another as this will be handle by the field type object and not by user code. You could have the first 2 fields per type handled by internal columns and the additional fields handled by a separate table. You could also have one main table for the main fields and a subtable with empty fields of different types for the extension. * Also you could add security to specific fields (make them only viewable by certain groups). * There are solutions to handle multiple entry fields in one integer field using a storage using binary combination (2^n + 2(n-1) + ..) and lookup tables between IDs and Strings. The could be transparent to the bugzilla code using the field type objects I've been talking about previously. - A separate table with the custom fields could be an interesting solution. For this to work the binary field with lookup table would need to be used for multiple entry fields and sufficient columns of each type would need to be created: * You would get only one line per bug. * This line does not need to be created if there are no custom fields in the bug. * You can query it with one join which would clearly makes things simpler. * You don't need to read it if you don't need the custom field data. * If really it is needed you could create a secondary line if you need additional custom fields (but that seems a little complicated). * You can do storage by type (id, numbers, strings, dates). To summarize, I think that bugzilla is way much more powerfull with custom fields. In terms of code it would be much more easy to program if the fields where abstracted inside a bug object (Bug.pm) for all features (currently you can use that for reading). Ludovic Gervase Markham wrote: > Over in > http://bugzilla.mozilla.org/show_bug.cgi?id=91037 > Dave said: > > "The current thinking is that we decide what the core types of these > fields are that we need to have available, and provide 3 or 4 of each > hardwired in the bugs table with a UI in the admin interface for > customizing the labels shown to the users or hiding/showing those > fields. This gives you a limited ability for adding custom fields, > while not creating headaches and performance issues with multiple joins > during queries." > > This solution seems to me to get the worst features of custom fields > without getting any of the good ones. We all know that > http://www.joelonsoftware.com/news/20020912.html > is why we are considering not having them. Joel argues that having > custom fields leads to people defining fields they don't really need, > field proliferation, and a a general muddle. The above plan doesn't > avoid that - it merely limits admins to 3 confusing email fields, 3 > confusing multi-selects and 3 confusing text fields. > > In addition, this not-quite-generalisation means our code will be > scattered with references to generic_text_field1, generic_multiselect2 > or whatever we decide to call these generic fields when we add them to > the bugs table. These will be carried around whether the user is using > them or not. > > An advantage given above is that general custom fields mean performance > issues with multiple joins during queries, and this method avoids that. > But this is thinking from an engineering point of view, not a usability > one. If this is a problem, then big installations can content themselves > with the default fields and do no joins. Or they can use mod_perl to > improve performance in other ways. Or get a better database, that does > lots of joins quicker. Or we could optimise our queries in a smarter way. > > Enough of this halfway house! :-) Let's have full custom fields, > implemented in a clean, generic and sensible way, or not at all. :-) > > Gerv > > ---- > To view or change your list settings, click here: > From ludovic at netvalue.com Tue Dec 10 20:38:13 2002 From: ludovic at netvalue.com (Ludovic Dubost) Date: Tue, 10 Dec 2002 21:38:13 +0100 Subject: New features to submit: Pager Notification, Response Template, MSProject Integration, Gantt Chart Message-ID: <3DF650B5.90805@netvalue.com> Hi, We have made some significant feature additions and I would like to know from you which one I should submit first to the bugzilla community (I know I have to do this using bugzilla.mozilla.org). Also, to be usable, these features need a little help for different reasons. One of them being that there are on a 2.14.1 codebase (non templatized) so there is some work to bring them back up-to-date but I think it can be worth it: - Notify Rules System The notify rules system is a way to trigger an additional email to any email address based on the matching of a saved query. For example, what can be done is create a query for your P1 bugs and configure it to send a mail to your mobile phone or pager. What will happen is that whenever a bug is updated that is assigned to you and that is a P1 priority the email will be sent out. This can be usefull to speed up the escalation process in help desk or any process handling using bugzilla. This code is a patch to process mail and some administration pages. It should not be to difficult to adapt. - Template Response System The template response system is a system to create response template automatically base on the comments in a bug and allow to find the best matching template using regular expression (giving points to each template when they match). Also this feature includes a system to send the comment by email to a Customer (we use a Customer field containing the email address) and save the comment in bugzilla at the same time. This allows us to handle our external help desk using bugzilla and bugzilla number but without the external user actually seeing bugzilla. The customer sends an email to a address and the mail is automatically loaded in bugzilla using the mail interface but without bugzilla responding to the mail (this is detected by the fact that the user does not have an account in the system). Then using the template system the operator can choose the best template and click on 'submit' to send the response, close the bug until the customer responds. Using the Bug number in the subject the response will reopen the bug automatically (we have made a patch for this). The operator can also modify the template in the comment area if necessary. This feature also supports a multi-langual mode using a 'language' custom fields. The mails for each language are loaded from a different email address and the language field is automatically stored and then the template detection system will only take the templates with the corresponding language. So this feature relies on a 'Customer' field that we have added a long time ago and on a 'language' custom field. - MS Project Integration and Gantt Chart Generation We have created an export and import feature for MS Project as well as a Gantt Chart generation module for project trees. - MS Project Export What this module allows to do is take a project tree (dependency tree) and export it in CSV. However it is not a basic export. It performs some specific MSProject conversion tasks. First, MSProject handled dependencies using numbers from 1 (and if you move a bug by one line all the dependencies are moving). This is really stupid but there is no way to get around it except feeding the information that MSProject expects. Also certain fields are converted (priority into a number from 0 to 1000). The export also relies on custom fields for time tracking. Currently I've not used the timetracking module that has been developped because I can't upgrade that easilly our bugzilla installation to 2.17 as we would have many patches to migrate. But I believe it could be easily adapted. Once you have exported a bug tree it is not possible to reimport it. The importing feature explained below would create new bugs. I plan to work on synching the data, starting with the timetracking data (workload, end date). - MS Project Import The MS Project Export allows to import a full MSProject plan into bugzilla, including workload and dates (provided that you have the right custom fields setup) and especially including dependencies and levels (in MS Project you can group bugs in subprojects): To have this working, I have created a custom field called 'Tracking Bug'. This field is inspired by the mozilla tracking bugs and corresponds in MSProject to a grouping of tasks (this allows to view a set of tasks a one big task). The limitation using bugzilla is that when you create a dependency between two tracking bugs this cannot be interpreted as a dependency (in MS Project you can do dependencies between tracking bugs). In order to do this I would have need to duplicate the dependency feature for tracking bugs and I believe it is not worth the trouble. The import feature is quite tricky because it is needed to reorder the bugs starting with ones have ne blockers. These bugs are inserted and the bug number retrieved and used to insert the next bugs. I've been testing as many things as possible before importing but it can still fail and if it does it stops where it is because I wasn't able to implement transactions (this did not seem to work). Even not perfect this feature is very powerfull if you want to use bugzilla to handle recurring projects. We use it to handle our production schedule of report (every month). It is especially interesting when the MS Project plan includes dates and workloads as it will allow the users to know when they have to finish their task. It gets very powerfull using the next feature: - Gantt Chart generation Using the Tracking Bug custom field and the time tracking custom fields (that could be adapted to use the new timetracking fields), I've used DBD::Chart and Date::Manip to create almost the exact same dependency chart as MS Project. As input for the dependency chart, it currently uses the dates save in the EstimatedEndDate and EstimatedWorkload custom fields. If there are no dates, the module will automatically show dependent tasks following each other. Currently there is a missing feature to not allow to tasks for the same person to be the same day (this would better represent certain project trees when dates are not missing and when the project contains many unrelated tasks). To build this feature DBD::Chart's gantt module has been significantly enhanced (it didn't allow to show different colors for the tracking bugs and it was not possible the order the bugs the way I decided). DBD::Manip has been used to handle workdays additions and substractions. Without this module the Gantt chart would have been impossible to build. I've attached an example PNG file of what is generated by the Gantt Chart module when dates and workload are correctly filled in (this is actual a real project plan that we use internally). Let me know what features would be the most interesting for the bugzilla community and if there are any people interesting into spending some time adapting the code to 2.17 and maybe enhancing the features to make them even more powerfull. Ludovic -------------- next part -------------- A non-text attachment was scrubbed... Name: 37115.png Type: image/png Size: 32612 bytes Desc: not available URL: From gerv at mozilla.org Tue Dec 10 21:10:15 2002 From: gerv at mozilla.org (Gervase Markham) Date: Tue, 10 Dec 2002 21:10:15 +0000 Subject: Custom Fields again In-Reply-To: <3DF64586.1030304@netvalue.com> References: <3DF62E83.70707@mozilla.org> <3DF64586.1030304@netvalue.com> Message-ID: <3DF65837.4070304@mozilla.org> If I may play devil's advocate for a moment... > Today we are using bugzilla for multiple areas: > > - bug tracking > - project management (tasks) > - internal help desk > - customer service > - pre-sales > - external help desk It can't be denied that if Bugzilla is to be used for all of these tasks, custom fields are a necessity. So we need to decide whether Bugzilla is for all of those things, or whether it's for tracking bugs in software and so should only have features related to that aim. Gerv From ludovic at netvalue.com Tue Dec 10 21:28:41 2002 From: ludovic at netvalue.com (Ludovic Dubost) Date: Tue, 10 Dec 2002 22:28:41 +0100 Subject: Custom Fields again In-Reply-To: <3DF65837.4070304@mozilla.org> References: <3DF62E83.70707@mozilla.org> <3DF64586.1030304@netvalue.com> <3DF65837.4070304@mozilla.org> Message-ID: <3DF65C89.3020402@netvalue.com> Gervase Markham wrote: > If I may play devil's advocate for a moment... > >> Today we are using bugzilla for multiple areas: >> >> - bug tracking >> - project management (tasks) >> - internal help desk >> - customer service >> - pre-sales >> - external help desk > > > It can't be denied that if Bugzilla is to be used for all of these > tasks, custom fields are a necessity. So we need to decide whether > Bugzilla is for all of those things, or whether it's for tracking bugs > in software and so should only have features related to that aim. > > Gerv I expected this comment and that's why I talked about the big benefit of using it for the different areas. The benefit being able to link all the issues together from the clients needs to the actual development. My understanding is that bugzilla.mozilla.org is also used for more that tracking bugs. It is actually used to manage the releases and the project itself to a certain extent (I'm refering to the mozilla 1.2 tracking bug used for releases or landing big patches). I understand that some of these features are CRM related and have nothing to do with bug tracking, but if you ask me, bugzilla has been a very powerfull help desk tool for us and was up to the challenge to handle the different needs we used it for. The benefit of have only one tool has been huge as we only needed to train the people in the company to one tool and everybody could follow their tasks from one tool. I'm not saying that bugzilla has to be used for all these features, but I'm definitely saying that it has the power to do so. This is what we have been doing for 2 years at our company and we saved ourselves: - one CRM system - two Help Desk System - one Project Management system (I mean a few MS Project licenses) - one bug tracking system However, I don't really mind if you don't integrate the features outside bug tracking in bugzilla as thanks to open source I can still add them myself. But I would say that it is a shame not to add these powerfull features when we can. Ludovic > > ---- > To view or change your list settings, click here: > From yannick.koehler at colubris.com Tue Dec 10 21:44:20 2002 From: yannick.koehler at colubris.com (Yannick Koehler) Date: Tue, 10 Dec 2002 16:44:20 -0500 Subject: Custom Fields Arguments Message-ID: <200212101644.21218.yannick.koehler@colubris.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sorry, it is attached because I first got an SMTP error. - -- Yannick Koehler -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE99mA0fuKOJNEyL1URAgL/AJ9Nt186RHlFvLPzqqPXm6JqC8nyGgCgopho v+yLTKJYnAxrAmISSIMm8oM= =k0qB -----END PGP SIGNATURE----- -------------- next part -------------- An embedded message was scrubbed... From: Yannick Koehler Subject: Re: Custom Fields again Date: Tue, 10 Dec 2002 16:40:14 -0500 Size: 3061 URL: From jon at vmware.com Tue Dec 10 22:59:24 2002 From: jon at vmware.com (Jonathan Schatz) Date: 10 Dec 2002 14:59:24 -0800 Subject: Custom Fields again In-Reply-To: <3DF62E83.70707@mozilla.org> References: <3DF62E83.70707@mozilla.org> Message-ID: <1039561164.6596.70.camel@jonschatz-lx.vmware.com> On Tue, 2002-12-10 at 10:12, Gervase Markham wrote: > Enough of this halfway house! :-) Let's have full custom fields, > implemented in a clean, generic and sensible way, or not at all. :-) why have a concept of "custom" fields in the first place? i mean, why not treat additional fields the same as normal fields? right now, the current implementation(s) of custom fields in 91037 involves a seperate table that contains custom field information. why not directly update the schema in bugs.bugs instead? adding a new custom field wouldn't require any more info than a normal field ( field, type, default).other than rebuilding / recompiling templates (which isn't an impossible problem), the biggest issue i see is searching. perhaps we could break out all of the sql query generation into a seperate module, which would be responsible for knowing which fields do and don't exist. i don't know, it's early (for me) and i haven't had a cup of coffee yet, but this is the idea i've been toying with in my head. i'll most likely move ahead with a prototype of this internally against 2.17.1 once i've ported my 2.16.1 changes there, so comments / flames are welcome. i just don't see the need to treat custom fields any differently than "normal" ones... -jon -- Jonathan Schatz Engineering System Administrator VMware, Inc "Te occidere possunt sed te edere non possunt nefas est." From justdave at syndicomm.com Tue Dec 10 23:07:25 2002 From: justdave at syndicomm.com (David Miller) Date: Tue, 10 Dec 2002 18:07:25 -0500 Subject: Custom Fields again In-Reply-To: <1039561164.6596.70.camel@jonschatz-lx.vmware.com> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> Message-ID: On 12/10/02 2:59 PM -0800, Jonathan Schatz wrote: > right now, the current implementation(s) of custom fields in 91037 > involves a seperate table that contains custom field information. why > not directly update the schema in bugs.bugs instead? adding a new custom > field wouldn't require any more info than a normal field ( field, type, > default).other than rebuilding / recompiling templates (which isn't an > impossible problem), the biggest issue i see is searching. perhaps we > could break out all of the sql query generation into a seperate module, > which would be responsible for knowing which fields do and don't exist. > > i don't know, it's early (for me) and i haven't had a cup of coffee yet, > but this is the idea i've been toying with in my head. i'll most likely > move ahead with a prototype of this internally against 2.17.1 once i've > ported my 2.16.1 changes there, so comments / flames are welcome. i just > don't see the need to treat custom fields any differently than "normal" > ones... This is actually a fair point. We already use LearnAboutColumns() to find out what the names of the columns are in the bugs table and just look for any field in the bug form with the same name to insert into a bug when creating it. A custom field (at least for creating the bug - querying is a separate issue) would simply be a matter of editing the enter_bug template to display it, and adding the column to the database. No code changes would even be required. I think the main reason nobody's really thought of that is because of the limit of 16 indexes on a table, and the bugs table already had that many. Well, we require MySQL 3.22.6 right now, for precisely that reason, because 3.22.6 and up allow more than 16 indexes on a table, so that's no longer an issue. Process_bug.cgi may do things a little more specifically as far as what fields are what, so that may take a little more work to be able to change a custom field on an existing bug, but creating a bug with custom fields in it is trivial. -- Dave Miller Project Leader, Bugzilla Bug Tracking System http://www.justdave.net/ http://www.bugzilla.org/ From gerv at mozilla.org Tue Dec 10 23:28:32 2002 From: gerv at mozilla.org (Gervase Markham) Date: Tue, 10 Dec 2002 23:28:32 +0000 Subject: Custom Fields again In-Reply-To: References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> Message-ID: <3DF678A0.6020804@mozilla.org> David Miller wrote: > We already use LearnAboutColumns() to find out what the names of the > columns are in the bugs table and just look for any field in the bug form > with the same name to insert into a bug when creating it. A custom field > (at least for creating the bug - querying is a separate issue) would simply > be a matter of editing the enter_bug template to display it, and adding the > column to the database. No code changes would even be required. Indeed. I assumed that adding extra columns to the bugs table was the way we were going to implement it; I didn't even consider that there might be another option. As I see it, it would work as follows: - When you define a new field, you define an alphanumeric name/tag (e.g. "buildid"), a description ("Build ID") and a type ("text"). - For single-select or text fields, we would then add an extra column to the bugs table of that type with that name. - For multi-select fields would instead have an extra table of that name, which mapped bug IDs to values. - show_bug would display all fields it could find in a sane way; for example, there'd be a vertical column of text fields, much like now, and multi-selects would be three across the page. - query.cgi would provide suitable query UI depending on field type. - Of course, the layout of both the above would be customisable; you would hard-code using a form name of the tag defined above, which fits with current practice of using DB column names for form names. Anyone else thinking along these lines? Gerv From bbaetz at student.usyd.edu.au Wed Dec 11 07:41:17 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Wed, 11 Dec 2002 18:41:17 +1100 Subject: Custom Fields again In-Reply-To: References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> Message-ID: <20021211074117.GA1294@mango.home> replying to both posts here... On Tue, Dec 10, 2002 at 06:07:25PM -0500, David Miller wrote: > On 12/10/02 2:59 PM -0800, Jonathan Schatz wrote: > > > right now, the current implementation(s) of custom fields in 91037 > > involves a seperate table that contains custom field information. why > > not directly update the schema in bugs.bugs instead? adding a new custom > > field wouldn't require any more info than a normal field ( field, type, > > default).other than rebuilding / recompiling templates (which isn't an > > impossible problem), the biggest issue i see is searching. perhaps we > > could break out all of the sql query generation into a seperate module, > > which would be responsible for knowing which fields do and don't exist. Well, yes, in theory. But how do you know what 'type' a field is? numeric, date, or what? Besides, what does this honestly give you that a 'real' implementation using a separate table doesn't? The only thing I could see someone arguing is search performance, and I think designing what would be a _massive_ hack arround mysql's join behaviour is a really large mistake. Not to mention that I'm not conviced that we couldn't do it with only a minor slowdown. > > > > i don't know, it's early (for me) and i haven't had a cup of coffee yet, > > but this is the idea i've been toying with in my head. i'll most likely > > move ahead with a prototype of this internally against 2.17.1 once i've > > ported my 2.16.1 changes there, so comments / flames are welcome. i just > > don't see the need to treat custom fields any differently than "normal" > > ones... > > This is actually a fair point. Correct, its a very good point, which I agree with. But see below.. > > We already use LearnAboutColumns() to find out what the names of the > columns are in the bugs table and just look for any field in the bug form > with the same name to insert into a bug when creating it. A custom field > (at least for creating the bug - querying is a separate issue) would simply > be a matter of editing the enter_bug template to display it, and adding the > column to the database. No code changes would even be required. LearnAboutColumns is a hack, which I want to disappear. Precisly because it is a hack (and a bit tricky to do cross-db, although it can be done) But thats not the only issue. You need to deal with: - fielddefs (which are also a hack, mind you) - bug_activity - querying - reporting/charting - other stuff I can't think of at the moment Now, Bug.pm has a list of 'available fields' as the patch of mine which is still waiting for review, so if everything used bug.pm things would be great (except for querying, which has special needs) > > I think the main reason nobody's really thought of that is because of the > limit of 16 indexes on a table, and the bugs table already had that many. > Yeah, and we require myiasm tables now because of that. The real reason people haven't been anxious to do it is that its ugly. > Well, we require MySQL 3.22.6 right now, for precisely that reason, because > 3.22.6 and up allow more than 16 indexes on a table, so that's no longer an > issue. > 3.23.6, you mean :) > Process_bug.cgi may do things a little more specifically as far as what > fields are what, so that may take a little more work to be able to change a > custom field on an existing bug, but creating a bug with custom fields in > it is trivial. Well, yes, but again, if we're going to all this trouble, why not formally define it? I mean, what do people have against a new table? We can then do stuff like multi-select values, and so on. The other advantage is that a lot of our current fields can then become user defined ones, simplifying the logic - we don't have to deal with status whiteboard and the url differently, since the update and search logic would be the same. Thered be a 5 line difference in teh templates to do teh linkification, and tahts done. Similarly for target_milestone vs os vs platform vs... Thats what I meant when I said that I agree that we should handle 'custom' fields the same as normal fields - see http://bugzilla.mozilla.org/show_bug.cgi?id=173127#c1 If we're going to do this (and I don't see why we shouldn't) why not do it properly? Bradley From bbaetz at student.usyd.edu.au Wed Dec 11 07:50:05 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Wed, 11 Dec 2002 18:50:05 +1100 Subject: Custom Fields again In-Reply-To: <3DF678A0.6020804@mozilla.org> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> Message-ID: <20021211075005.GB1294@mango.home> On Tue, Dec 10, 2002 at 11:28:32PM +0000, Gervase Markham wrote: > Indeed. I assumed that adding extra columns to the bugs table was the > way we were going to implement it; I didn't even consider that there > might be another option. It didn't occur to me that anyone would be trying to dynamically create tables on the fly.... What if you called your custom field 'requests' or something? (Yes, we can prefix with custfield_, I suppose) > > As I see it, it would work as follows: > - When you define a new field, you define an alphanumeric name/tag (e.g. > "buildid"), a description ("Build ID") and a type ("text"). > - For single-select or text fields, we would then add an extra column to > the bugs table of that type with that name. > - For multi-select fields would instead have an extra table of that > name, which mapped bug IDs to values. > - show_bug would display all fields it could find in a sane way; for > example, there'd be a vertical column of text fields, much like now, and > multi-selects would be three across the page. > - query.cgi would provide suitable query UI depending on field type. > - Of course, the layout of both the above would be customisable; you > would hard-code using a form name of the tag defined above, which fits > with current practice of using DB column names for form names. > > Anyone else thinking along these lines? Almost. What we do is define three new tables: fields --------------------------------- id | auto_increment/serial/etc name | varchar(x) type | int - mapping to const in Bugzilla::Constants field_vals --------------------------------- id | auto_increment/serial/etc fieldid | references fields.id value | varchar(x) field_entries --------------------------------- fieldid | references fields.id bug_id | referencs bugs.bug_id value | varchar(x) And, umm.... Thats it. This gives us: - multiple values-per bug for multi-select fields - constraints for popup types, via field_vals - id mappings (value is actually an integer, except for teh freeform text types) - consistency + resuse of search code - just change the fieldid val in the WHERE clause See my coomments in the bug (as well as the existing patches) for more. Bradley From gerv at mozilla.org Wed Dec 11 09:22:57 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 11 Dec 2002 09:22:57 +0000 Subject: Custom Fields again In-Reply-To: <20021211075005.GB1294@mango.home> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> Message-ID: <3DF703F1.3040008@mozilla.org> Bradley Baetz wrote: > On Tue, Dec 10, 2002 at 11:28:32PM +0000, Gervase Markham wrote: > >>Indeed. I assumed that adding extra columns to the bugs table was the >>way we were going to implement it; I didn't even consider that there >>might be another option. > > It didn't occur to me that anyone would be trying to dynamically create > tables on the fly.... What if you called your custom field 'requests' or > something? (Yes, we can prefix with custfield_, I suppose) What's wrong with "dynamically" creating tables? And what's more "dynamic" about them than checksetup.pl's creation of tables? (Yes, we could prefix table names with cf_ to avoid clashes.) > Almost. What we do is define three new tables: > > fields > --------------------------------- > id | auto_increment/serial/etc > name | varchar(x) > type | int - mapping to const in Bugzilla::Constants > > field_vals > --------------------------------- > id | auto_increment/serial/etc > fieldid | references fields.id > value | varchar(x) > > field_entries > --------------------------------- > fieldid | references fields.id > bug_id | referencs bugs.bug_id > value | varchar(x) > > And, umm.... Thats it. This gives us: > > - multiple values-per bug for multi-select fields > - constraints for popup types, via field_vals > - id mappings (value is actually an integer, except for teh freeform > text types) > - consistency + resuse of search code - just change the fieldid val in > the WHERE clause ...and a whacking great performance hit, surely. You have to join to one or more tables for every custom field you have. Also, numeric data is not numeric, it's varchar, and that's bad for search performance. If we add rows to the bugs table for each (single-valued) field, this has several advantages. For a start, without too much work, we could make many of our current bugs tables fields into custom ones, allowing admins to remove those they don't like, because their existence will be the same as other custom fields. It also lets us use numbers for numeric data, and other appropriate SQL types - including short varchars for short data, and long varchars for long data. And I am certain search performance will be better without all those joins. We can still have an external table with the names and types of the custom fields, if that makes things better. It would be nice if we could just scan all the column names from the bugs table, but maybe there are reasons why that wouldn't work. Gerv From bbaetz at student.usyd.edu.au Wed Dec 11 12:13:32 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Wed, 11 Dec 2002 23:13:32 +1100 Subject: Custom Fields again In-Reply-To: <3DF703F1.3040008@mozilla.org> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> Message-ID: <20021211121332.GA2503@mango.home> On Wed, Dec 11, 2002 at 09:22:57AM +0000, Gervase Markham wrote: > > What's wrong with "dynamically" creating tables? And what's more > "dynamic" about them than checksetup.pl's creation of tables? (Yes, we > could prefix table names with cf_ to avoid clashes.) Firstly, its going to be massivly slow. How big is bmo's bugs table, again? That has to be copied for each update to the schema. What happens when teh webserver times out half way through? > > >Almost. What we do is define three new tables: > ...and a whacking great performance hit, surely. You have to join to one > or more tables for every custom field you have. Also, numeric data is > not numeric, it's varchar, and that's bad for search performance. It also gives us per product fields, which is something which is often requested, although I guess we can do that. Yes, you have to do an extra table join. As I said, I don't want mysql's suckiness to influence this - we can do the subselect emulation like we do for product and component. The multi value stuff then just becomes a single lookup, with no joins involved. numeric/varchar is sucky, I agree, though. > > If we add rows to the bugs table for each (single-valued) field, this > has several advantages. For a start, without too much work, we could > make many of our current bugs tables fields into custom ones, allowing > admins to remove those they don't like, because their existence will be > the same as other custom fields. It also lets us use numbers for numeric > data, and other appropriate SQL types - including short varchars for > short data, and long varchars for long data. And I am certain search > performance will be better without all those joins. See above. > > We can still have an external table with the names and types of the > custom fields, if that makes things better. It would be nice if we could > just scan all the column names from the bugs table, but maybe there are > reasons why that wouldn't work. It won't work. :) How do you know if a varchar is a string, or a url, or whatever? You also need the constraint table, remember, for mapping names to ids for popup stuff (single or multivalued) I dunno, maybe this isn't as bad as I'm making out. There are setups where the bugzilla user doesn't have schema privs, but we don't support that anyway. This would make it harder to do so, though. Bradley From justdave at syndicomm.com Wed Dec 11 13:10:41 2002 From: justdave at syndicomm.com (David Miller) Date: Wed, 11 Dec 2002 08:10:41 -0500 Subject: Custom Fields again In-Reply-To: <20021211121332.GA2503@mango.home> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> Message-ID: On 12/11/02 11:13 PM +1100, Bradley Baetz wrote: > On Wed, Dec 11, 2002 at 09:22:57AM +0000, Gervase Markham wrote: >> >> What's wrong with "dynamically" creating tables? And what's more >> "dynamic" about them than checksetup.pl's creation of tables? (Yes, we >> could prefix table names with cf_ to avoid clashes.) > > Firstly, its going to be massivly slow. How big is bmo's bugs table, > again? That has to be copied for each update to the schema. What happens > when teh webserver times out half way through? Why do it from the web? Adding a custom field is either an initial configuration thing or it's a Big Deal to add when you do it. Use a command-line utility in the shell to do it (just like checksetup.pl) or even use a data file (localconfig?) to declare what should be there, and have checksetup.pl do that when it runs. Choosing which products it's valid for and what the multi-select values are for multi-select fields could certainly be done from the web interface however. -- Dave Miller Project Leader, Bugzilla Bug Tracking System http://www.justdave.net/ http://www.bugzilla.org/ From bbaetz at student.usyd.edu.au Wed Dec 11 13:18:52 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Thu, 12 Dec 2002 00:18:52 +1100 Subject: Custom Fields again In-Reply-To: References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> Message-ID: <20021211131852.GA2921@mango.home> On Wed, Dec 11, 2002 at 08:10:41AM -0500, David Miller wrote: > > Why do it from the web? Well, fair enough, although we currently let everyhting else happen from the web. > Adding a custom field is either an initial > configuration thing or it's a Big Deal to add when you do it. Why is it a Big Deal? Or rather, why should it be a Big Deal? Bradley From justdave at syndicomm.com Wed Dec 11 13:22:01 2002 From: justdave at syndicomm.com (David Miller) Date: Wed, 11 Dec 2002 08:22:01 -0500 Subject: Custom Fields again In-Reply-To: <20021211131852.GA2921@mango.home> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <20021211131852.GA2921@mango.home> Message-ID: On 12/12/02 12:18 AM +1100, Bradley Baetz wrote: >> Adding a custom field is either an initial >> configuration thing or it's a Big Deal to add when you do it. > > Why is it a Big Deal? Or rather, why should it be a Big Deal? Because you're a) changing the schema b) changing your developer's bug tracking process Both of those things should only be done with careful consideration and you should Really Mean It when you do it. -- Dave Miller Project Leader, Bugzilla Bug Tracking System http://www.justdave.net/ http://www.bugzilla.org/ From bbaetz at student.usyd.edu.au Wed Dec 11 13:37:11 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Thu, 12 Dec 2002 00:37:11 +1100 Subject: Custom Fields again In-Reply-To: References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <20021211131852.GA2921@mango.home> Message-ID: <20021211133711.GA3019@mango.home> On Wed, Dec 11, 2002 at 08:22:01AM -0500, David Miller wrote: > On 12/12/02 12:18 AM +1100, Bradley Baetz wrote: > > >> Adding a custom field is either an initial > >> configuration thing or it's a Big Deal to add when you do it. > > > > Why is it a Big Deal? Or rather, why should it be a Big Deal? > > Because you're > a) changing the schema No, thats backwards :) I agre that changing the schema is a Big Deal (which is part of the reason I don' think that this is a good idea), but that doesn't imply that adding a new field should be a Big Deal - that argument would have you using a cmdline process even if we go with my linked tables scheme. > b) changing your developer's bug tracking process > Not necessarily. It fact, not usually. You're going to be adding the ability to add more information to the bug, and thats about it. How does adding a field for the os, or the platform, or for the phase of the moon, require a potentially long downtime? Bradley From justdave at syndicomm.com Wed Dec 11 13:45:01 2002 From: justdave at syndicomm.com (David Miller) Date: Wed, 11 Dec 2002 08:45:01 -0500 Subject: Custom Fields again In-Reply-To: <20021211133711.GA3019@mango.home> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <20021211131852.GA2921@mango.home> <20021211133711.GA3019@mango.home> Message-ID: On 12/12/02 12:37 AM +1100, Bradley Baetz wrote: > No, thats backwards :) I agre that changing the schema is a Big Deal > (which is part of the reason I don' think that this is a good idea), but > that doesn't imply that adding a new field should be a Big Deal - that > argument would have you using a cmdline process even if we go with my > linked tables scheme. OK, so we fork off a background process, trap the signals so Apache can't kill it, close STDIN, STDOUT, and STDERR so Apache will disconnect from the browser, and put a meta refresh in the page that reloads another page every 10 seconds that tracks the progress of the first process. It'd require some careful checking to make sure that process was the only one that could be tracked that way to avoid security issues though. -- Dave Miller Project Leader, Bugzilla Bug Tracking System http://www.justdave.net/ http://www.bugzilla.org/ From chicks at chicks.net Wed Dec 11 14:55:43 2002 From: chicks at chicks.net (Christopher Hicks) Date: Wed, 11 Dec 2002 09:55:43 -0500 (EST) Subject: Custom Fields again In-Reply-To: <20021211133711.GA3019@mango.home> Message-ID: On Thu, 12 Dec 2002, Bradley Baetz wrote: > No, thats backwards :) I agre that changing the schema is a Big Deal > (which is part of the reason I don' think that this is a good idea), but > that doesn't imply that adding a new field should be a Big Deal All of this worry leads me to feel like we need a better way to manage the schema. (Others have touched on this.) A variety of schemes exist to do this. It /shouldn't/ be a big deal for people to add (or drop) fields and for the software to dynamically figure it out. Beyond primary keys and foreign keys there shouldn't be that much that's sacred. Ultimately, I'd like to see the 'core' schema split into the sacred and optional bits. Then local additions and choices regarding the optional core pieces can be stored seperately. I'd recommend looking at Alzabo for managing the schema. We have been using an internal system to manage regularly changing schemas and the meta-information about each field. I'm probably going to migrate all of that onto Alzabo during our next major development cycle. > > b) changing your developer's bug tracking process > > Not necessarily. It fact, not usually. You're going to be adding the > ability to add more information to the bug, and thats about it. How does > adding a field for the os, or the platform, or for the phase of the > moon, require a potentially long downtime? It was mentioned in the URL against custom fields early in this discussion that the more fields that were added the less likely bugs were to be submitted. A number of uses have been mentioned that indicate this isn't real since for a variety of reasons the bug submitters may never be exposed to any of these fields. In our installation we almost totally ignore the two platform fields even for our software-oriented bugs since what we're doing is web-oriented and we rarely have any platform-specific issues. I'd love to get rid of them, but given the choice of making changes that will make upgrading harder and ignoring them, it seems easier to just ignore them. So, the lack of clean customization seems to impede our bug tracking process more so than customizability causing problems. -- Programming is a Dark Art, and it will always be. The programmer is fighting against the two most destructive forces in the universe: entropy and human stupidity. They're not things you can always overcome with a "methodology" or on a schedule. -Damian Conway, Perl God From VFedrushkov at chel.LUKoil.com Wed Dec 11 13:44:18 2002 From: VFedrushkov at chel.LUKoil.com (VFedrushkov at chel.LUKoil.com) Date: Wed, 11 Dec 2002 18:44:18 +0500 Subject: Custom Fields again Message-ID: Good $daytime, > From: Bradley Baetz [mailto:bbaetz at student.usyd.edu.au] > Sent: Wednesday, December 11, 2002 6:19 PM > To: developers at bugzilla.org > Subject: Re: Custom Fields again > > Adding a custom field is either an initial > > configuration thing or it's a Big Deal to add when you do it. > Why is it a Big Deal? Or rather, why should it be a Big Deal? Because we don't want custom fields to be 'cheap'. Instead we encourage administrator to think well before doing that. And yes, if custom fields reconfiguration would require me to take database offline, I'd first look how to implement desired feature within standard functionality. Regards, Willy. -- No easy hope or lies | Vitaly "Willy the Pooh" Fedrushkov Shall bring us to our goal, | Control Systems and Processes Division But iron sacrifice | LUKOIL Company, Chelyabinsk Branch Of Body, Will and Soul. | VFedrushkov at LUKoil.com R.Kipling | +7 351 262 0367 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bbaetz at student.usyd.edu.au Wed Dec 11 14:10:40 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Thu, 12 Dec 2002 01:10:40 +1100 Subject: Custom Fields again In-Reply-To: References: <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <20021211131852.GA2921@mango.home> <20021211133711.GA3019@mango.home> Message-ID: <20021211141040.GA3161@mango.home> On Wed, Dec 11, 2002 at 08:45:01AM -0500, David Miller wrote: > > OK, so we fork off a background process, trap the signals so Apache can't > kill it, close STDIN, STDOUT, and STDERR so Apache will disconnect from the > browser, and put a meta refresh in the page that reloads another page every > 10 seconds that tracks the progress of the first process. It'd require > some careful checking to make sure that process was the only one that could > be tracked that way to avoid security issues though. ... and have that work on windows, too :) My original point wasn't that a command line interface was inherantly bad, but rather that the need for it was a reason not to use separate tables. I'm not sure if I still feel that way, mind you. Bradley From bbaetz at student.usyd.edu.au Wed Dec 11 14:13:16 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Thu, 12 Dec 2002 01:13:16 +1100 Subject: Custom Fields again In-Reply-To: References: Message-ID: <20021211141316.GA3223@mango.home> On Wed, Dec 11, 2002 at 06:44:18PM +0500, VFedrushkov at chel.LUKoil.com wrote: > > Why is it a Big Deal? Or rather, why should it be a Big Deal? > > Because we don't want custom fields to be 'cheap'. Instead we > encourage administrator to think well before doing that. Think of what? If someone wants a field, then, well, they want one :) > And yes, if custom fields reconfiguration would require me to take > database offline, I'd first look how to implement desired feature > within standard functionality. Well, the point is that with custfields this will become standard functionality. OTOH, I'm not sure how long it would take to change the schema, even on bmo - even with extrenal tables we still have to insert an entry for every row (for some types, at least), and thats not cheap. Bradley From bbaetz at student.usyd.edu.au Wed Dec 11 14:18:44 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Thu, 12 Dec 2002 01:18:44 +1100 Subject: Custom Fields again In-Reply-To: References: <20021211133711.GA3019@mango.home> Message-ID: <20021211141844.GA3234@mango.home> On Wed, Dec 11, 2002 at 09:55:43AM -0500, Christopher Hicks wrote: > All of this worry leads me to feel like we need a better way to manage the > schema. (Others have touched on this.) A variety of schemes exist to do > this. It /shouldn't/ be a big deal for people to add (or drop) fields and > for the software to dynamically figure it out. Beyond primary keys and > foreign keys there shouldn't be that much that's sacred. Ultimately, I'd > like to see the 'core' schema split into the sacred and optional bits. > Then local additions and choices regarding the optional core pieces can be > stored seperately. Well, we can find out whats in the schema. Its doing something useful with it which is the problem. If I know that I have a VARCHAR(x) column called 'myField', how do I know what its meant to contain? We really don't want to encourage people to make their own updates to teh schema manually. > > I'd recommend looking at Alzabo for managing the schema. We have been > using an internal system to manage regularly changing schemas and the > meta-information about each field. I'm probably going to migrate all of > that onto Alzabo during our next major development cycle. A lot of the stff in checksetup is more than just creating the fields - we run conversion code to move stuff arround, too. To an extent, teh code in checksetup is a manual version of that, which tests for existance of certain fields, and then does 'stuff' to them > It was mentioned in the URL against custom fields early in this discussion > that the more fields that were added the less likely bugs were to be > submitted. I don't remember seeing that, and I don't know if its true. To some extent, it may be - if I can limit my search for possible duplicates to my specific version/os/etc, then I'm more likely to find the correct one. OTOH, that argument doesn't hold for something like mozilla, which is almost entirely cross platform code, wherejsut because it was reported on windows doesn't mean that it won't happen on the mac. > A number of uses have been mentioned that indicate this isn't > real since for a variety of reasons the bug submitters may never be > exposed to any of these fields. In our installation we almost totally > ignore the two platform fields even for our software-oriented bugs since > what we're doing is web-oriented and we rarely have any platform-specific > issues. I'd love to get rid of them, but given the choice of making > changes that will make upgrading harder and ignoring them, it seems easier > to just ignore them. So, the lack of clean customization seems to impede > our bug tracking process more so than customizability causing problems. > Right, and custfields will let you remove that sort of stuff. Bradley From chicks at chicks.net Wed Dec 11 15:19:31 2002 From: chicks at chicks.net (Christopher Hicks) Date: Wed, 11 Dec 2002 10:19:31 -0500 (EST) Subject: Custom Fields again In-Reply-To: Message-ID: On Wed, 11 Dec 2002 VFedrushkov at chel.LUKoil.com wrote: > Because we don't want custom fields to be 'cheap'. Instead we > encourage administrator to think well before doing that. I'm all for encouraging people to think before they do things, but the perceived harm of making it easy seems like FUD. If we can make the process less painful on the development side, what harm is there in letting people add fields to see if they'll be of any use? -- Programming is a Dark Art, and it will always be. The programmer is fighting against the two most destructive forces in the universe: entropy and human stupidity. They're not things you can always overcome with a "methodology" or on a schedule. -Damian Conway, Perl God From bugreport at peshkin.net Wed Dec 11 15:28:52 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Wed, 11 Dec 2002 07:28:52 -0800 Subject: Custom Fields again References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> Message-ID: <3DF759B4.1060903@peshkin.net> David Miller wrote: > Why do it from the web? Adding a custom field is either an initial > configuration thing or it's a Big Deal to add when you do it. Use a > command-line utility in the shell to do it (just like checksetup.pl) or > even use a data file (localconfig?) to declare what should be there, and > have checksetup.pl do that when it runs. Choosing which products it's > valid for and what the multi-select values are for multi-select fields > could certainly be done from the web interface however > Using localconfig solves several problems. If I add a field using localconfig and run checksetup to get it into the DB, it clears up many migration issues that apply to fields added by sites using checksetup. Once a field is added using localconfig, then it should be possible for me to use the web interface to specify how that field is processed and displayed OR I should be able to use the web interface to specify how that field is processed and customize templates myself to handle the display activity. This gets us the best of both worlds. Someone who only touches localconfig and the web can use custom fields. People who customize templates can get a bit more mileage out of it. From chicks at chicks.net Wed Dec 11 20:17:03 2002 From: chicks at chicks.net (Christopher Hicks) Date: Wed, 11 Dec 2002 15:17:03 -0500 (EST) Subject: Custom Fields again In-Reply-To: <20021211141844.GA3234@mango.home> Message-ID: On Thu, 12 Dec 2002, Bradley Baetz wrote: > Well, we can find out whats in the schema. Its doing something useful > with it which is the problem. If I know that I have a VARCHAR(x) column > called 'myField', how do I know what its meant to contain? We really > don't want to encourage people to make their own updates to teh schema > manually. That's why having a standard way to manage field-level meta-information is critical. A varchar() doesn't pose as much ambiguity in my experience as a numeric column. Is it financial? Are we storing 5 decimal places for auditing purposes, but only displaying 2 in most cases? Regardless of the column type a number of questions arise: Is it something that should be editable by anyone? Is it a virtual field? Should it be displayed in HTML as an INPUT, TEXTAREA, or SELECT? If it's a SELECT how are the coded values chosen and how should the titles for those selections be displayed? How wide a column should be used for displaying in reports? Should the underlying code or the SELECT's title value be used for display in reports? What is the title to appear for editing? What is the column heading for reports? An effective meta-information infrastructure is the only way to deal with this without losing your head. We define a set of defaults for each column type, but from experience at least 80% of fields have their defaults overridden by some degree. Usually this is just fixing the title for reports since the sql field name is rarely pretty, but I'd hate to have to explicitly answer all of the above questions for every field in a database with 50 tables and some tables that have 80 columns. [shiver.] Several folks have mentioned using seperate tables to store the custom fields. I doubt this would be of much use in most cases. It seems that if people choose to have optional fields that they're likely to want them to apply to most of the bugs in their system. But regardless of that if you choose to make a record in the 'custom fields table' itself optional on whether any of those fields is used then every query against the DB is going to require an outer join which would slow all interaction with the bugs table and complicate coding considerably. The biggest thing that encourages someone to want this extra table for custom fields is storage space, but the difficulties of the approach seem much more expensive that some additional disk space. Plus, regardless of those practical considerations, having tables with a one-to-one relation violates normalization. -- Programming is a Dark Art, and it will always be. The programmer is fighting against the two most destructive forces in the universe: entropy and human stupidity. They're not things you can always overcome with a "methodology" or on a schedule. -Damian Conway, Perl God From gerv at mozilla.org Wed Dec 11 22:50:56 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 11 Dec 2002 22:50:56 +0000 Subject: Custom Fields again In-Reply-To: References: Message-ID: <3DF7C150.2070300@mozilla.org> > I'm all for encouraging people to think before they do things, but the > perceived harm of making it easy seems like FUD. If we can make the > process less painful on the development side, what harm is there in > letting people add fields to see if they'll be of any use? I agree. If custom fields is a feature to be used with care, we should say so in the documentation, not by making it intentionally more inconvenient than it could be. Gerv From gerv at mozilla.org Wed Dec 11 22:52:31 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 11 Dec 2002 22:52:31 +0000 Subject: Custom Fields again In-Reply-To: <3DF759B4.1060903@peshkin.net> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <3DF759B4.1060903@peshkin.net> Message-ID: <3DF7C1AF.2000200@mozilla.org> Joel Peshkin wrote: > Using localconfig solves several problems. If I add a field using > localconfig and run checksetup to get it into the DB, it clears up many > migration issues that apply to fields added by sites using checksetup. I don't understand what you mean here. Could you elaborate? Gerv From gerv at mozilla.org Wed Dec 11 22:58:47 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 11 Dec 2002 22:58:47 +0000 Subject: Custom Fields again In-Reply-To: References: Message-ID: <3DF7C327.2090401@mozilla.org> Christopher Hicks wrote: > An effective meta-information infrastructure is the only way to deal with > this without losing your head. True; bbaetz has convinced me that we need a table which gives meta-information for the custom fields - tag, type, default and so on. We can't just look for all the column names. > to apply to most of the bugs in their system. But regardless of that if > you choose to make a record in the 'custom fields table' itself optional > on whether any of those fields is used then every query against the DB is > going to require an outer join which would slow all interaction with the > bugs table and complicate coding considerably. The biggest thing that > encourages someone to want this extra table for custom fields is storage > space, but the difficulties of the approach seem much more expensive that > some additional disk space. Plus, regardless of those practical > considerations, having tables with a one-to-one relation violates > normalization. I think these are important points. So, I'm still thinking that, controlled by a metadata table, single-valued fields should be added to the bugs table, and multi-valued fields should be stored in a table with a name corresponding to the tag. I'd say the majority of fields are single-valued, so this cuts down on the number of joins. Yes, it has the disadvantage that you store a value for a custom field for every bug even if you are using it in one product. I think that's a price worth paying - on b.m.o., for example, with its 2.5Gb database, as I understand it the bugs table is under 100Mb. The vast majority of the 2.5Gb is attachments. But I'm sure Myk will correct me if I'm wrong. Whether this is all controlled from the web or checksetup is an implementation detail, IMO, and can be discussed later. Gerv From gerv at mozilla.org Wed Dec 11 23:48:42 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 11 Dec 2002 23:48:42 +0000 Subject: New features to submit: Pager Notification, Response Template, In-Reply-To: <3DF650B5.90805@netvalue.com> References: <3DF650B5.90805@netvalue.com> Message-ID: <3DF7CEDA.9010108@mozilla.org> Ludovic Dubost wrote: > We have made some significant feature additions and I would like to know > from you which one I should submit first to the bugzilla community (I > know I have to do this using bugzilla.mozilla.org). Thank you very much for the offers :-) > - Notify Rules System > > What will happen is that whenever a bug is updated that is assigned to > you and that is a P1 priority the email will be sent out. How is this implemented? The obvious implementation surely has performance prolems... > - Template Response System I'm not convinced of the general utility of this one... > - MS Project Integration and Gantt Chart Generation > > We have created an export and import feature for MS Project as well as a > Gantt Chart generation module for project trees. > > - MS Project Export > > What this module allows to do is take a project tree (dependency tree) > and export it in CSV. In the new world, this would be another template for showdependencytree.cgi. > - MS Project Import > > The MS Project Export allows to import a full MSProject plan into > bugzilla, including workload and dates (provided that you have the right > custom fields setup) I have a feeling that, because it requires some of your custom fields, that this would be a bit specialist. > - Gantt Chart generation > > Using the Tracking Bug custom field and the time tracking custom fields > (that could be adapted to use the new timetracking fields), I've used > DBD::Chart and Date::Manip to create almost the exact same dependency > chart as MS Project. These look very cool; in our world, they'd probably be based on the timetracking fields and a dependency tree. > To build this feature DBD::Chart's gantt module has been significantly > enhanced (it didn't allow to show different colors for the tracking bugs > and it was not possible the order the bugs the way I decided). Have you sent these changes to the maintainer of that module? Again, thanks for letting us know about these :-) Gerv From myk at mozilla.org Wed Dec 11 23:53:24 2002 From: myk at mozilla.org (Myk Melez) Date: Wed, 11 Dec 2002 15:53:24 -0800 Subject: Custom Fields again In-Reply-To: <3DF7C327.2090401@mozilla.org> References: <3DF7C327.2090401@mozilla.org> Message-ID: <3DF7CFF4.2090700@mozilla.org> Gervase Markham wrote: > I think that's a price worth paying - on b.m.o., for example, with its > 2.5Gb database, as I understand it the bugs table is under 100Mb. The > vast majority of the 2.5Gb is attachments. But I'm sure Myk will > correct me if I'm wrong. Close. b.m.o is now closer to 4GB, of which 2.5GB is attachments. The bugs table, with indexes, is 50MB. -myk From gerv at mozilla.org Thu Dec 12 00:18:05 2002 From: gerv at mozilla.org (Gervase Markham) Date: Thu, 12 Dec 2002 00:18:05 +0000 Subject: Custom Fields again In-Reply-To: <3DF7CFF4.2090700@mozilla.org> References: <3DF7C327.2090401@mozilla.org> <3DF7CFF4.2090700@mozilla.org> Message-ID: <3DF7D5BD.4080303@mozilla.org> Myk Melez wrote: > Gervase Markham wrote: > >> I think that's a price worth paying - on b.m.o., for example, with its >> 2.5Gb database, as I understand it the bugs table is under 100Mb. The >> vast majority of the 2.5Gb is attachments. But I'm sure Myk will >> correct me if I'm wrong. > > Close. b.m.o is now closer to 4GB, of which 2.5GB is attachments. The > bugs table, with indexes, is 50MB. Just for our information, what's the rest? Comments? Gerv From bugreport at peshkin.net Thu Dec 12 01:19:59 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Wed, 11 Dec 2002 17:19:59 -0800 Subject: Custom Fields again References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <3DF759B4.1060903@peshkin.net> <3DF7C1AF.2000200@mozilla.org> Message-ID: <3DF7E43F.9060906@peshkin.net> Gervase Markham wrote: > Joel Peshkin wrote: > >> Using localconfig solves several problems. If I add a field using >> localconfig and run checksetup to get it into the DB, it clears up >> many migration issues that apply to fields added by sites using >> checksetup. > > > I don't understand what you mean here. Could you elaborate? Certainly. Currently, if I make my own site-specific DB fields in checksetup, I am itching for a merge conflict when I update. If I define what I want to have added to the "stock" DB using localconfig, then I use a standard (unhacked) checksetup and I wont have a big merge mess when checksetup starts to be database-agnostic (because I haven't tweaked it there). After all, my schema changes "belong" in exactlt the same place in checksetup as the very next schema change done to the tip will edit. I am guaranteed a conflict when I update. From willy at lukoil.uu.ru Thu Dec 12 05:02:18 2002 From: willy at lukoil.uu.ru (Vitaly Fedrushkov) Date: Thu, 12 Dec 2002 10:02:18 +0500 (YEKT) Subject: Custom Fields again In-Reply-To: <20021211141844.GA3234@mango.home> Message-ID: Good $daytime, > Date: Thu, 12 Dec 2002 01:18:44 +1100 > From: Bradley Baetz > To: developers at bugzilla.org > Subject: Re: Custom Fields again > > All of this worry leads me to feel like we need a better way to > > manage the schema. (Others have touched on this.) A variety of > > schemes exist to do this. It /shouldn't/ be a big deal for people > > to add (or drop) fields and for the software to dynamically figure > > it out. Beyond primary keys and foreign keys there shouldn't be > > that much that's sacred. Ultimately, I'd like to see the 'core' > > schema split into the sacred and optional bits. Then local > > additions and choices regarding the optional core pieces can be > > stored seperately. > Well, we can find out whats in the schema. Its doing something > useful with it which is the problem. If I know that I have a > VARCHAR(x) column called 'myField', how do I know what its meant to > contain? Having our own metadata is inevitable, because: o Native DBMS data types denote nothing but storage format. Data types we're talking about here often denote semantics. o Native DBMS data types aren't portable and can't be used in Bugzilla UI code. o Field names must not be used anywhere in UI. At least in localizable world. Values for multiple-choice fields must not be used either, but that's another story. o The few 'sacred' fields are those which (a) form database schema, and (b) define bug lifecycle. All others are all about entry and search, and people may want to turn them off. o There are entities with many-to-many relation to fields and products, notably enable flags (to supersede has_xxxx configuration parameters) and fixed field values. If implemented using three extra tables, the VARCHAR can be easily resolved by having 'numvalue', 'strvalue', 'datevalue' and perhaps 'blobvalue' fields in values table. Not so savvy w.r.t. storage, but better performance as no conversion needed. BUT: The problem with three extra tables is that there's almost no method by which it can be effectively optimized. The problem with schema changes is smaller: difficulty with nonscalar custom fields. > A lot of the stff in checksetup is more than just creating the > fields - we run conversion code to move stuff arround, too. To an > extent, teh code in checksetup is a manual version of that, which > tests for existance of certain fields, and then does 'stuff' to them Not having Web tools for things like schema changes is not bad. Sometimes custom fields will require database tuning, additional indexes, capacity planning, etc. Here comes the Database Administrator, hopefully armed with more powerful tools than just a browser. Notice about sixteen indices is not so important because anyone with this problem will then move to, say, Oracle -- aren't we going towards portable Bugzilla? Regards, Willy. -- No easy hope or lies | Vitaly "Willy the Pooh" Fedrushkov Shall bring us to our goal, | Control Systems and Processes Division But iron sacrifice | LUKOIL Company, Chelyabinsk Branch Of Body, Will and Soul. | willy at lukoil.uu.ru +7 3512 620367 R.Kipling | VVF1-RIPE From ludovic at netvalue.com Thu Dec 12 08:42:29 2002 From: ludovic at netvalue.com (Ludovic Dubost) Date: Thu, 12 Dec 2002 09:42:29 +0100 Subject: New features to submit: Pager Notification, Response Template, In-Reply-To: <3DF7CEDA.9010108@mozilla.org> References: <3DF650B5.90805@netvalue.com> <3DF7CEDA.9010108@mozilla.org> Message-ID: <3DF84BF5.1040902@netvalue.com> Gervase Markham wrote: > Ludovic Dubost wrote: > >> We have made some significant feature additions and I would like to >> know from you which one I should submit first to the bugzilla >> community (I know I have to do this using bugzilla.mozilla.org). > > > Thank you very much for the offers :-) > >> - Notify Rules System >> >> What will happen is that whenever a bug is updated that is assigned >> to you and that is a P1 priority the email will be sent out. > > > How is this implemented? The obvious implementation surely has > performance prolems... In order to limit the Notify Rules treatment time, first only the assignee rules are run for each bug that is updated. Then the query is a concatenation of the saved query + bug_id = $bug_id which limits the query to the bug being analyzed. Because of this, I think that if they are performance issues they would only be if a user has two many rules. I work on a day to day basis with 3 rules and have never had any problems. > >> - Template Response System > > > > I'm not convinced of the general utility of this one... This is interesting when bugzilla is too complex to understand for a part of the population you want to interact using bugzilla. This is typically a 'Help Desk' feature interesting in very specific cases.. But as it was said in another comment, 'help-desk' is not the primary usage of bugzilla.. > >> - MS Project Integration and Gantt Chart Generation >> >> We have created an export and import feature for MS Project as well >> as a Gantt Chart generation module for project trees. >> >> - MS Project Export >> >> What this module allows to do is take a project tree (dependency >> tree) and export it in CSV. > > > In the new world, this would be another template for > showdependencytree.cgi. Well yes and no.. There are some differences in the way the bugs are organized (using the Tracking Bug field). Also if you want to be able to import the bugs in MS Project you need to recreate the dependencies in a language that MS Project understand (instead of 38765 <-> 38785 it should be 1 <-> 2 because the bug numbers correspond to the line in the MS Project bug table and not to a arbitrary number so unless you export your whole database it won't work). However, the feature is linked from a modified version of the showdependencytree.cgi page. > >> - MS Project Import >> >> The MS Project Export allows to import a full MSProject plan into >> bugzilla, including workload and dates (provided that you have the >> right custom fields setup) > > > I have a feeling that, because it requires some of your custom fields, > that this would be a bit specialist. Most of the custom fields are the timetracking fields and this can be adapted to the bugzilla implementation. In any case, the use of the custom fields in the import can be disabled quite easily. The custom fields are necessary for the Gantt chart feature. > >> - Gantt Chart generation >> >> Using the Tracking Bug custom field and the time tracking custom >> fields (that could be adapted to use the new timetracking fields), >> I've used DBD::Chart and Date::Manip to create almost the exact same >> dependency chart as MS Project. > > > These look very cool; in our world, they'd probably be based on the > timetracking fields and a dependency tree. Yes.. The use of the 'tracking bug' field allows to replicate the organisation of the bugs by subprojects and this is very readable.. I think it would be interesting to generalize a difference between a dependency (where time is involved) and a grouping of bugs (tracking bugs). This allows to present the dependency tree differently. > >> To build this feature DBD::Chart's gantt module has been >> significantly enhanced (it didn't allow to show different colors for >> the tracking bugs and it was not possible the order the bugs the way >> I decided). > > > Have you sent these changes to the maintainer of that module? This is planned.. The code has less than 1 week.. Ludovic From myk at mozilla.org Thu Dec 12 20:20:04 2002 From: myk at mozilla.org (Myk Melez) Date: Thu, 12 Dec 2002 12:20:04 -0800 Subject: Custom Fields again In-Reply-To: <3DF7CFF4.2090700@mozilla.org> References: <3DF7C327.2090401@mozilla.org> <3DF7CFF4.2090700@mozilla.org> Message-ID: <3DF8EF74.6030402@mozilla.org> Myk Melez wrote: > Gervase Markham wrote: > >> I think that's a price worth paying - on b.m.o., for example, with >> its 2.5Gb database, as I understand it the bugs table is under 100Mb. >> The vast majority of the 2.5Gb is attachments. But I'm sure Myk will >> correct me if I'm wrong. > > > Close. b.m.o is now closer to 4GB, of which 2.5GB is attachments. > The bugs table, with indexes, is 50MB. Addendum: the shadow log is about 750MB, and the comments are about 500MB. Everything else fits in the last 200MB. -myk From bbaetz at student.usyd.edu.au Fri Dec 13 06:53:59 2002 From: bbaetz at student.usyd.edu.au (Bradley Baetz) Date: Fri, 13 Dec 2002 17:53:59 +1100 Subject: Custom Fields again In-Reply-To: <3DF8EF74.6030402@mozilla.org> References: <3DF7C327.2090401@mozilla.org> <3DF7CFF4.2090700@mozilla.org> <3DF8EF74.6030402@mozilla.org> Message-ID: <20021213065359.GA1213@mango.home> On Thu, Dec 12, 2002 at 12:20:04PM -0800, Myk Melez wrote: > Myk Melez wrote: > > Addendum: the shadow log is about 750MB, and the comments are about > 500MB. Everything else fits in the last 200MB. Err. Are you telling me that the shadowlog table is 750 megs? Something's sersiously wrong - it should have been way under that. Like 1Mb at most, modulo added attachments. Anyway, you can TRUNCATE it now, since you're not using it (my step 0.5 for removing shadowdb support didn't do that, and the final patch drops the table entirely). It doesn't hurt, unless you turn shadowdb support back on without clearing it.. /me checks synshadowdb source Hmm, most of that may be empty - syncshadowdb was using DELETE instead of TRUNCATE (which wasn't available when it was originally written). 750Mb still sounds massivly big to me Bradley From bugreport at peshkin.net Fri Dec 13 14:46:34 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Fri, 13 Dec 2002 06:46:34 -0800 Subject: fine whine - input needed Message-ID: <3DF9F2CA.2030905@peshkin.net> All: The whining system is seriously in need of a rewrite. I have an initial cut of a new version (fine_whine.pl) at http://bugzilla.mozilla.org/show_bug.cgi?id=185090 that sends formatted whines and moves most decisions to a template. Before embarking on a full rewrite, how do we want whining to work? Should users have prefs for whining? Should the same whining cron be able to send differrent reports weekly and daily? Should watchers get whinemail? Please comment here or in that bug. -Joel From myk at mozilla.org Sat Dec 14 00:06:37 2002 From: myk at mozilla.org (Myk Melez) Date: Fri, 13 Dec 2002 16:06:37 -0800 Subject: Custom Fields again In-Reply-To: <20021213065359.GA1213@mango.home> References: <3DF7C327.2090401@mozilla.org> <3DF7CFF4.2090700@mozilla.org> <3DF8EF74.6030402@mozilla.org> <20021213065359.GA1213@mango.home> Message-ID: <3DFA760D.4040808@mozilla.org> Bradley Baetz wrote: >Err. Are you telling me that the shadowlog table is 750 megs? > > > -rw-rw---- 1 mysql other 782075644 Dec 8 03:24 shadowlog.MYD Remember that I haven't been resetting it every night for months. -myk From gerv at mozilla.org Sun Dec 15 10:34:08 2002 From: gerv at mozilla.org (Gervase Markham) Date: Sun, 15 Dec 2002 10:34:08 +0000 Subject: Custom Fields again In-Reply-To: <3DF7E43F.9060906@peshkin.net> References: <3DF62E83.70707@mozilla.org> <1039561164.6596.70.camel@jonschatz-lx.vmware.com> <3DF678A0.6020804@mozilla.org> <20021211075005.GB1294@mango.home> <3DF703F1.3040008@mozilla.org> <20021211121332.GA2503@mango.home> <3DF759B4.1060903@peshkin.net> <3DF7C1AF.2000200@mozilla.org> <3DF7E43F.9060906@peshkin.net> Message-ID: <3DFC5AA0.6010203@mozilla.org> Joel Peshkin wrote: > Gervase Markham wrote: > >> Joel Peshkin wrote: >> >>> Using localconfig solves several problems. If I add a field using >>> localconfig and run checksetup to get it into the DB, it clears up >> >> I don't understand what you mean here. Could you elaborate? > > Certainly. > > Currently, if I make my own site-specific DB fields in checksetup, I am Ah, here's the misunderstanding. This is not a choice of "localconfig or checksetup" - I agree that getting people to edit checksetup is bad. But I think editing localconfig isn't much better. I think new bug fields could be defined using the Web UI, because there's no reason for them not to be. checksetup never needs to know about them, except in the most general terms. Gerv From jon at vmware.com Wed Dec 18 22:33:12 2002 From: jon at vmware.com (Jonathan Schatz) Date: 18 Dec 2002 14:33:12 -0800 Subject: os metadata / getting rid of enums Message-ID: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> At my workplace, we use the OS field in a very detailed way. Currently, our list of OS's is around 100 entries (every version + variant of major linux distributions, every windows version + patchlevel since windows for workgroups, etc). One of the most requested features we have is the ability to search for generic OS's, ie searching for new bugs on "linux" would return all bugs filed against redhat 7.2, suse 8.0, etc. I've started to implement this in our private bugzilla branch, and I've realized that this would be easier for me if the OS field was broken out of its' current enum container and into its' own table. In that table i'd have boolean columns (ie, is_linux, is_windows, etc). this brings up 2 questions: 1) Is there a reason why all of the enum columns in bugs.bugs haven't been broken out into their own tables? Isn't this a key in making bugzilla database independent? 2) Would others have use for generic OS searching as described above? We've currently got a rather unusual setup in that we use 2 OS fields for bugs. If others would find this useful, I'd write the code against HEAD and patch it to work with our setup; if not, I'd go ahead and write it for our own use. feedback / flames are welcome. -jon -- Jonathan Schatz Engineering System Administrator VMware, Inc "Te occidere possunt sed te edere non possunt nefas est." From gerv at mozilla.org Wed Dec 18 22:45:06 2002 From: gerv at mozilla.org (Gervase Markham) Date: Wed, 18 Dec 2002 22:45:06 +0000 Subject: os metadata / getting rid of enums In-Reply-To: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> Message-ID: <3E00FA72.9020806@mozilla.org> Jonathan Schatz wrote: > At my workplace, we use the OS field in a very detailed way. Currently, > our list of OS's is around 100 entries (every version + variant of major > linux distributions, every windows version + patchlevel since windows > for workgroups, etc). One of the most requested features we have is the > ability to search for generic OS's, ie searching for new bugs on "linux" > would return all bugs filed against redhat 7.2, suse 8.0, etc. The obvious thing to do here, it seems to me, is to have two fields. One has the values Linux, Windows (NT), Windows (consumer), BSD and the other has the detailed breakdown. You use JS to keep the two fields consistent in the interface. The second possible thing to do is make sure all the Linux ones contain the string "Linux", and the Windows ones contain the string "Win", and do regexp searching on the field instead of "exact match". You could customise the query page to make this possible. > 1) Is there a reason why all of the enum columns in bugs.bugs haven't > been broken out into their own tables? Isn't this a key in making > bugzilla database independent? I believe that this is on the cards, because enums aren't cross-database. Gerv From bugreport at peshkin.net Wed Dec 18 22:42:11 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Wed, 18 Dec 2002 14:42:11 -0800 Subject: os metadata / getting rid of enums References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> Message-ID: <3E00F9C3.3010503@peshkin.net> Jonathan Schatz wrote: > > > I've >realized that this would be easier for me if the OS field was broken out >of its' current enum container and into its' own table. In that table >i'd have boolean columns (ie, is_linux, is_windows, etc). this brings up >2 questions: > >2) Would others have use for generic OS searching as described above? >We've currently got a rather unusual setup in that we use 2 OS fields >for bugs. If others would find this useful, I'd write the code against >HEAD and patch it to work with our setup; if not, I'd go ahead and write >it for our own use. > > > I think that having a way of doing this for Both OS and Hardware would be very useful. A bug logged against a product running on the Pentium3 may effect all the versions on x86-class machines or be very specific to the P3. I certainly have analogous issues. I would vote for doing this for Both OS and Hardware. -Joel (The real one) From bugreport at peshkin.net Wed Dec 18 22:48:33 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Wed, 18 Dec 2002 14:48:33 -0800 Subject: os metadata / getting rid of enums References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> <3E00F9C3.3010503@peshkin.net> Message-ID: <3E00FB41.5090400@peshkin.net> As Gerv points out, doing this with a strict hierarchy can be done by regexp. But... when this is something that wants to be effectively a multiple select, is there a broader need for it? -Joel From jon at vmware.com Wed Dec 18 23:06:28 2002 From: jon at vmware.com (Jonathan Schatz) Date: 18 Dec 2002 15:06:28 -0800 Subject: os metadata / getting rid of enums In-Reply-To: <3E00FB41.5090400@peshkin.net> References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> <3E00F9C3.3010503@peshkin.net> <3E00FB41.5090400@peshkin.net> Message-ID: <1040252788.14603.33.camel@jonschatz-lx.vmware.com> On Wed, 2002-12-18 at 14:48, Joel Peshkin wrote: > As Gerv points out, doing this with a strict hierarchy can be done by > regexp. it could, but it doesn't seem as clean. and it wouldn't work in every case. Shouldn't OSX show up under "BSD"? Also, we'd have to start using OS names like this: Red Hat 7.2 Linux Solaris 2.5.1 SysV OSX 10.2 BSD etc. I think this would be a quick fix, but not necessarily address the issues i'm having. > But... when this is something that wants to be effectively a multiple > select, is there a broader need for it? I think there is. You mentioned the arch field, which is another good example of this. Selecting "x86" should match "Athlon XP 2100", "VIA C3 800", and "Pentium 4 2.8ghz w/ HT". This isn't particularly complex code to write, and doesn't really require that many changes overall. off the top of my head, we'd need to change how @legal_opsys is defined, change Bugzilla::Search to comprehend these "effective multiple selects", add the op_sys table, and add edit[opsys|arch].cgi to allow maintenance. I'll probably have this finished in a day or two here, so for now i'll do this against cvs and send the results out. there are a few bugs that could have relevance here, but none seem to specifically address the os or arch fields: http://bugzilla.mozilla.org/show_bug.cgi?id=17453 http://bugzilla.mozilla.org/show_bug.cgi?id=146104 http://bugzilla.mozilla.org/show_bug.cgi?id=94534 http://bugzilla.mozilla.org/show_bug.cgi?id=97706 -jon -- Jonathan Schatz Engineering System Administrator VMware, Inc "Te occidere possunt sed te edere non possunt nefas est." From chicks at chicks.net Thu Dec 19 00:08:01 2002 From: chicks at chicks.net (Christopher Hicks) Date: Wed, 18 Dec 2002 19:08:01 -0500 (EST) Subject: os metadata / getting rid of enums In-Reply-To: <3E00FA72.9020806@mozilla.org> Message-ID: On Wed, 18 Dec 2002, Gervase Markham wrote: > The obvious thing to do here, it seems to me, is to have two fields. One > has the values Linux, Windows (NT), Windows (consumer), BSD and the > other has the detailed breakdown. You use JS to keep the two fields > consistent in the interface. > > The second possible thing to do is make sure all the Linux ones contain > the string "Linux", and the Windows ones contain the string "Win", and > do regexp searching on the field instead of "exact match". You could > customise the query page to make this possible. But neither of these give you everything that Jonathan's table solution does. For instance, you might want to track which versions of Windows are fully 32-bit or support USB. By having a table that's expandable you let the database drive the app which is a good thing IMHO. -- Programming is a Dark Art, and it will always be. The programmer is fighting against the two most destructive forces in the universe: entropy and human stupidity. They're not things you can always overcome with a "methodology" or on a schedule. -Damian Conway, Perl God From jon at vmware.com Thu Dec 19 02:27:45 2002 From: jon at vmware.com (Jonathan Schatz) Date: 18 Dec 2002 18:27:45 -0800 Subject: os metadata / getting rid of enums In-Reply-To: <1040252788.14603.33.camel@jonschatz-lx.vmware.com> References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> <3E00F9C3.3010503@peshkin.net> <3E00FB41.5090400@peshkin.net> <1040252788.14603.33.camel@jonschatz-lx.vmware.com> Message-ID: <1040264864.14608.62.camel@jonschatz-lx.vmware.com> turns out there's a bug specifically opened on this thread's topic: http://bugzilla.mozilla.org/show_bug.cgi?id=12309 i'm going to hack together a patch for this over the holidays here, so i'll have something in a week or so. just for clarity: should i submit code here first for feedback (as opposed to attaching a patch to a bug)? -jon -- Jonathan Schatz Engineering System Administrator VMware, Inc "Te occidere possunt sed te edere non possunt nefas est." From bugreport at peshkin.net Thu Dec 19 02:37:16 2002 From: bugreport at peshkin.net (Joel Peshkin) Date: Wed, 18 Dec 2002 18:37:16 -0800 Subject: os metadata / getting rid of enums References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> <3E00F9C3.3010503@peshkin.net> <3E00FB41.5090400@peshkin.net> <1040252788.14603.33.camel@jonschatz-lx.vmware.com> <1040264864.14608.62.camel@jonschatz-lx.vmware.com> Message-ID: <3E0130DC.2050705@peshkin.net> Jonathan Schatz wrote: > >just for clarity: should i submit code here first for feedback (as >opposed to attaching a patch to a bug)? > >-jon > > Attaching a patch to a bug is a good next step. I'd suggest attaching an early version (and indicating that it is one) and asking for feedback early in the process. Your first few patches often wind up getting a suprisingly detailed critique. Until you get the hang of what the reviewers look for, it can be a bit of a shocker. You could ask for the early critique on IRC or here. -Joel From jon at vmware.com Fri Dec 20 02:53:35 2002 From: jon at vmware.com (Jonathan Schatz) Date: 19 Dec 2002 18:53:35 -0800 Subject: os metadata / getting rid of enums In-Reply-To: <3E0130DC.2050705@peshkin.net> References: <1040250792.14608.12.camel@jonschatz-lx.vmware.com> <3E00F9C3.3010503@peshkin.net> <3E00FB41.5090400@peshkin.net> <1040252788.14603.33.camel@jonschatz-lx.vmware.com> <1040264864.14608.62.camel@jonschatz-lx.vmware.com> <3E0130DC.2050705@peshkin.net> Message-ID: <1040352815.8419.42.camel@jonschatz-lx.vmware.com> On Wed, 2002-12-18 at 18:37, Joel Peshkin wrote: > You could ask for the early critique on IRC or here. alright, here goes. the patch is at http://divisionbyzero.com/op_sys.patch and is against cvs HEAD. this does the following: -adds a op_sys table -removes the enum op_sys column in bugs -adds a op_sys_id column in bugs (which points to op_sys.id) checksetup.pl will create + populate the new table for you. everything i've tried (new bugs, queries, changing, reports) has worked fine with this code. feedback is requested. -jon -- Jonathan Schatz Engineering System Administrator VMware, Inc "Te occidere possunt sed te edere non possunt nefas est." From geeta.tripathi at bhartitelesoft.com Fri Dec 6 13:07:28 2002 From: geeta.tripathi at bhartitelesoft.com (geeta) Date: Fri, 6 Dec 2002 18:37:28 +0530 Subject: The Issues on Bugzilla 2.16.1 References: Message-ID: <004101c29d28$7200c160$800310ac@tripathi.bhartitelesoft.com> Hello All, Can anybody please help me getting the following error when trying to run the query.page data/versioncache did not return a true value at globals.pl line 627. main::GetVersionTable() called at /home/httpd/html/bugzilla-2.16.1/userp refs.cgi line 367 [Thu Feb 6 18:25:36 2003] [error] [client 172.16.3.74] Premature end of script headers: /home/httpd/html/bugzilla-2.16.1/userprefs.cgi data/versioncache did not return a true value at globals.pl line 627 (#1) (F) A required (or used) file must return a true value to indicate that it compiled correctly and ran its initialization code correctly. It's traditional to end such a file with a "1;", though any true value would do. See perlfunc/require. Uncaught exception from user code: data/versioncache did not return a true value at globals.pl line 627. main::GetVersionTable() called at /home/httpd/html/bugzilla-2.16.1/query .cgi line 190 [Thu Feb 6 18:26:09 2003] [error] [client 172.16.3.128] Premature end of script headers: /home/httpd/html/bugzilla-2.16.1/query.cgi Looking forward to hear from you. Thanks and regards, Geeta -------------- next part -------------- An HTML attachment was scrubbed... URL: