From fredrik at jonson.org Wed Nov 2 16:31:03 2011 From: fredrik at jonson.org (Fredrik Jonson) Date: Wed, 02 Nov 2011 11:31:03 -0500 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 Message-ID: Hi all, I've resently upgraded from cgi to mod_perl on a bz 4.0.2 installation that use postgresql and UTF-8. I soon found that the quicksearch doesn't work when you submit non-ascii characters, like ???. You get a internal server error and the log states: "ERROR: invalid byte sequence for encoding "UTF8": 0xe5e4f6" I believe that you a similar issue on landfill with the query below: https://landfill.bugzilla.org/bugzilla-tip-pg/buglist.cgi?quicksearch=%E5%E4%F6 The patch below fixes that error on my local installation. Haven't tested it beyond that. I'd be happy to open a issue if it is of general interest. RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/Search/Quicksearch.pm,v retrieving revision 1.35.2.7 diff -u -r1.35.2.7 Quicksearch.pm --- Bugzilla/Search/Quicksearch.pm 27 Jan 2011 13:03:39 -0000 1.35.2.7 +++ Bugzilla/Search/Quicksearch.pm 2 Nov 2011 16:08:32 -0000 @@ -30,6 +30,8 @@ use Bugzilla::Field; use Bugzilla::Util; +use Encode::Encoder; + use List::Util qw(min max); use List::MoreUtils qw(firstidx); @@ -138,6 +140,8 @@ $and = 0; $or = 0; + $searchstring = Encode::encode('UTF-8', $searchstring); + # Remove leading and trailing commas and whitespace. $searchstring =~ s/(^[\s,]+|[\s,]+$)//g; ThrowUserError('buglist_parameters_required') unless ($searchstring); -- Fredrik Jonson _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From lpsolit at gmail.com Wed Nov 2 16:44:55 2011 From: lpsolit at gmail.com (=?UTF-8?B?RnLDqWTDqXJpYyBCdWNsaW4=?=) Date: Wed, 02 Nov 2011 17:44:55 +0100 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 In-Reply-To: References: Message-ID: <4EB17387.4040008@gmail.com> Le 02. 11. 11 17:31, Fredrik Jonson a ?crit : > https://landfill.bugzilla.org/bugzilla-tip-pg/buglist.cgi?quicksearch=%E5%E4%F6 Maybe this is a bug in old versions of PostgreSQL. The version running on landfill is 8.3.11. I cannot reproduce the problem with MySQL, SQLite and PostgreSQL 9.0.5. LpSolit From lpsolit at gmail.com Wed Nov 2 16:49:44 2011 From: lpsolit at gmail.com (=?UTF-8?B?RnLDqWTDqXJpYyBCdWNsaW4=?=) Date: Wed, 02 Nov 2011 17:49:44 +0100 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 In-Reply-To: References: Message-ID: <4EB174A8.4050503@gmail.com> Le 02. 11. 11 17:31, Fredrik Jonson a ?crit : > https://landfill.bugzilla.org/bugzilla-tip-pg/buglist.cgi?quicksearch=%E5%E4%F6 Ah, the reason for landfill-tip-pg is that it's not running with the utf8 parameter enabled. LpSolit From fredrik at jonson.org Wed Nov 2 19:47:51 2011 From: fredrik at jonson.org (Fredrik Jonson) Date: Wed, 02 Nov 2011 14:47:51 -0500 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 References: Message-ID: LpSolit wrote: > > https://landfill.bugzilla.org/bugzilla-tip-pg/buglist.cgi?quicksearch=%E5%E4%F6 > > Ah, the reason for landfill-tip-pg is that it's not running with the > utf8 parameter enabled. Any chance someone can enable utf8 on tip-pg, just to make sure that's really the culprit in the above case on landfill? The interesting part is that landfill/tip-pg really currently behaves just like my own installation did befor the patch; the quicksearch forms are not able to search for ???, while the forms on the query.cgi page are able to. https://landfill.bugzilla.org/bugzilla-tip-pg/buglist.cgi?content=%C3%A5%C3%A4%C3%B6 https://landfill.bugzilla.org/bugzilla-tip-pg/query.cgi?content=%E5%E4%F6 -- Fredrik Jonson _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From fredrik at jonson.org Wed Nov 2 19:59:16 2011 From: fredrik at jonson.org (Fredrik Jonson) Date: Wed, 02 Nov 2011 14:59:16 -0500 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 References: Message-ID: LpSolit wrote: > Maybe this is a bug in old versions of PostgreSQL. The version running > on landfill is 8.3.11. I cannot reproduce the problem with MySQL, SQLite > and PostgreSQL 9.0.5. I am running postgres 9.1, the database encoding is UTF-8, and it is deployed on a fresh standard install of Debian Squeeze 6.0.3. I haven't done any particular configuration of apache2 apart from installing mod_perl, enabling mod_env, mod_headers, and mod_expires. It may very well be a configuration issue, but I believe I've followed the installation instructions in the bugzilla documentation to the point. While they do not go into details, I can't recall seeing any encoding specific change required for running under mod_perl. LpSolit, as you're unable to recreate the issue on your systems, can you recall if you have any encoding related settings in apache or postgresql that I might have missed and that would explain why quicksearch works for you? -- Fredrik Jonson _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From lpsolit at gmail.com Wed Nov 2 20:05:36 2011 From: lpsolit at gmail.com (=?ISO-8859-1?Q?Fr=E9d=E9ric_Buclin?=) Date: Wed, 02 Nov 2011 21:05:36 +0100 Subject: Quicksearch, mod_perl, postgresql, and UTF-8 In-Reply-To: References: Message-ID: <4EB1A290.8040703@gmail.com> Le 02. 11. 11 20:59, Fredrik Jonson a ?crit : > LpSolit, as you're unable to recreate the issue on your systems, can you > recall if you have any encoding related settings in apache or postgresql > that I might have missed and that would explain why quicksearch works for > you? Besides running recode.pl when checksetup.pl asked me to do so? Nothing comes to mind, no. :) But in your case, it's a fresh installation, not an upgrade, so the utf8 parameter was already enabled when you installed Bugzilla. So I don't know what's going on. LpSolit From user at mozilla-xp.com/ Sun Nov 6 22:45:05 2011 From: user at mozilla-xp.com/ (Blackburn31Roslyn) Date: Sun, 06 Nov 2011 16:45:05 -0600 Subject: Bugzilla extensions galore References: Message-ID: freelance writer _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From dmarshal at yahoo-inc.com Fri Nov 11 16:15:36 2011 From: dmarshal at yahoo-inc.com (David Marshall) Date: Fri, 11 Nov 2011 08:15:36 -0800 Subject: Yahoo! bug_id 5000000 Message-ID: It only took us about 9 months to go from 4000000 to 5000000. +---------+---------------------+ | bug_id | creation_ts | +---------+---------------------+ | 5000000 | 2011-11-10 22:20:31 | +---------+---------------------+ From lpsolit at gmail.com Fri Nov 11 20:14:22 2011 From: lpsolit at gmail.com (=?ISO-8859-1?Q?Fr=E9d=E9ric_Buclin?=) Date: Fri, 11 Nov 2011 21:14:22 +0100 Subject: Yahoo! bug_id 5000000 In-Reply-To: References: Message-ID: <4EBD821E.4030101@gmail.com> Le 11. 11. 11 17:15, David Marshall a ?crit : > +---------+---------------------+ > | bug_id | creation_ts | > +---------+---------------------+ > | 5000000 | 2011-11-10 22:20:31 | > +---------+---------------------+ Congratulations!! :) Nice to see that Bugzilla still works fine with so many bugs. LpSolit From knocte at NOSPAMPLEASEgmail.com Wed Nov 16 13:35:34 2011 From: knocte at NOSPAMPLEASEgmail.com (Andres G. Aragoneses) Date: Wed, 16 Nov 2011 13:35:34 +0000 Subject: Milestone for OpenID? Message-ID: <7JKdnSyqR53AIF7TnZ2dnUVZ_ridnZ2d@mozilla.org> Hey guys, Saw some days ago that someone posted a fairly simple patch for OpenID support. Any maintainer has an idea if this can be reviewed and merged sooner rather than later? It would be great. Thanks _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From gerv at mozilla.org Thu Nov 17 10:20:03 2011 From: gerv at mozilla.org (Gervase Markham) Date: Thu, 17 Nov 2011 10:20:03 +0000 Subject: Milestone for OpenID? In-Reply-To: <7JKdnSyqR53AIF7TnZ2dnUVZ_ridnZ2d@mozilla.org> References: <7JKdnSyqR53AIF7TnZ2dnUVZ_ridnZ2d@mozilla.org> Message-ID: <2OCdnUxW-c9OQlnTnZ2dnUVZ_qSdnZ2d@mozilla.org> On 16/11/11 13:35, Andres G. Aragoneses wrote: > Hey guys, > > Saw some days ago that someone posted a fairly simple patch for OpenID > support. Where? :-) > Any maintainer has an idea if this can be reviewed and merged > sooner rather than later? It would be great. Is there a bug? Gerv _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From lpsolit at gmail.com Thu Nov 17 16:14:34 2011 From: lpsolit at gmail.com (=?ISO-8859-1?Q?Fr=E9d=E9ric_Buclin?=) Date: Thu, 17 Nov 2011 17:14:34 +0100 Subject: Milestone for OpenID? In-Reply-To: <2OCdnUxW-c9OQlnTnZ2dnUVZ_qSdnZ2d@mozilla.org> References: <7JKdnSyqR53AIF7TnZ2dnUVZ_ridnZ2d@mozilla.org> <2OCdnUxW-c9OQlnTnZ2dnUVZ_qSdnZ2d@mozilla.org> Message-ID: <4EC532EA.2000908@gmail.com> Le 17. 11. 11 11:20, Gervase Markham a ?crit : >> Saw some days ago that someone posted a fairly simple patch for OpenID >> support. > > Where? :-) I guess here: https://bugzilla.mozilla.org/show_bug.cgi?id=294608 LpSolit From dkl at mozilla.com Thu Nov 17 17:10:56 2011 From: dkl at mozilla.com (David Lawrence) Date: Thu, 17 Nov 2011 12:10:56 -0500 Subject: Milestone for OpenID? In-Reply-To: <4EC532EA.2000908@gmail.com> References: <7JKdnSyqR53AIF7TnZ2dnUVZ_ridnZ2d@mozilla.org> <2OCdnUxW-c9OQlnTnZ2dnUVZ_qSdnZ2d@mozilla.org> <4EC532EA.2000908@gmail.com> Message-ID: <4EC54020.4030203@mozilla.com> On 11/17/2011 11:14 AM, Fr?d?ric Buclin wrote: > Le 17. 11. 11 11:20, Gervase Markham a ?crit : >>> Saw some days ago that someone posted a fairly simple patch for OpenID >>> support. >> Where? :-) > I guess here: https://bugzilla.mozilla.org/show_bug.cgi?id=294608 > I feel like adoption of this would be quicker if it was written as an extension instead of in the core code. For an example of something similar, Gerv has implemented BrowserID support as an extension. https://bugzilla.mozilla.org/show_bug.cgi?id=672841 This would also depend on https://bugzilla.mozilla.org/show_bug.cgi?id=698418 getting committed which adds the necessary hooks. If I could find the time, I would be willing to do this dkl -- David Lawrence dkl at mozilla.com