FW: Bugzilla 5.0.6 REST interface fails after renaming table groups to groeps
Ignace Danneels
ignace.danneels at tremec.com
Fri Sep 1 11:37:37 UTC 2023
Additional info. I did have to hide some project specific information, I hope you understand:
I put a few breakpoint in an attempt to get more details (sorry for the large screenshots, because I also wanted you to see the local variables):
Python script calls:
[cid:image007.png at 01D9DCD7.FBDFD550]
A little later xmlrpclib sends the request:
[cid:image008.png at 01D9DCD9.7430F600]
The total request_body is:
[cid:image009.png at 01D9DCD9.7430F600]
The python library than parses the response
[cid:image010.png at 01D9DCD9.7430F600]
This is the value of “data”
[cid:image005.png at 01D9DCD7.4A7BD890]
Kind regards,
Ignace Danneels
Controls Design Engineer
TREMEC
Office: +32 (0) 50 977 766
From: Ignace Danneels
Sent: Friday, 1 September 2023 12:02
To: Subject: RE: Bugzilla 5.0.6 REST interface fails after renaming table groups to groeps
After the python scripts makes a connection to the Bugzilla server, it creates an xmlrpc interface.
Then the scripts basically does the following:
The post-commit script merely does the following:
[cid:image003.png at 01D9DCCA.0FD5BAB0]
The error I get when executing the last line is
[cid:image004.png at 01D9DCCB.BEC8C200]
It would appear the bz_bug assignment is working, because I can get some info, such as the id, but other queries, such as requesting the history generate an error (similar to the add_comment error above
I get the following error:
[cid:image006.png at 01D9DCCC.2339CD10]
If there’s a log in Bugzilla that would hold additional information, let me know and I’ll try to report it to you.
Kind regards,
Ignace Danneels
Controls Design Engineer
TREMEC
Office: +32 (0) 50 977 766
From: David Miller <justdave at bugzilla.org<mailto:justdave at bugzilla.org>>
Sent: Friday, 1 September 2023 11:26
To: Ignace Danneels <ignace.danneels at tremec.com<mailto:ignace.danneels at tremec.com>>; support-list at bugzilla.org<mailto:support-list at bugzilla.org>
Subject: Re: Bugzilla 5.0.6 REST interface fails after renaming table groups to groeps
[CYBERSECURITY DESC] MENSAJE DE UN REMITENTE EXTERNO / MESSAGE FROM AN EXTERNAL SENDER
Toma precauciones extremas, no hagas click en los enlaces.
Take extreme precautions, do not click on unknown links.
We should have the MySQL 8 compatibility landed at the latest by the end of November (but probably much sooner). We gave some good patches for it now, they just need review and testing.
Version 5.2 should be coming out this weekend. It still won't have the MySQL 8 compatibility, but we will be landing it on that branch once it's ready. 5.0.x will not be getting that fix as it'll be considered too invasive for an older branch. But since you already upgraded beyond 5.0.4, you're essentially on the 5.2 version already (because 5.0.5 made invasive changes that it shouldn't have, and should have been 5.2 with those changes).
As far as your commit message goes... what do you mean by "no longer works"? Are you getting an error? If so, what's the error? Is anything showing up in the logs on either end of the transaction?
On 9/1/23 5:06 AM, Ignace Danneels wrote:
Hello Dave,
I hope I can bother you one more time.
As explained in earlier mails, we have renamed the table ‘groups’ to ‘groeps’ in order to be compliant with MySQL 8.
(I still don’t see a Bugzilla release that is compatible with it)
We have changed all references in the Bugzilla perl scripts and from a webinterface part, everything is working as expected.
We do run into one issue, for which I may need your help.
I would need to get in touch with the Bugzilla REST specialist.
The issue is the following:
We are working with subversion and when we make a commit, then we have a post commit hook that runs a python script to also commit the comment from the subversion commit into the Bugzilla issue(s).
The python script uses the python library bugzillatools. (https://pypi.org/project/bugzillatools/)
Some history:
We had Bugzilla 5.0.3 before, when transitioning to a new server, we also upgraded Bugzilla to 5.0.6
The post-commit hook did work when we had BZ 5.0.3. It no longer works after installing 5.0.6
In the original 5.0.3, we had made changes (only) to the following files:
[cid:image001.png at 01D9DCC9.BD5BAF60]
When Installing 5.0.6, we made changes to the following files (changing ‘groups’ to ‘groeps’)
[cid:image002.png at 01D9DCC9.BD5BAF60]
If you want, I am prepared to share the modified files with you.
Unfortunately, I’m not very familiar with REST.
As you can see, in 5.0.3, we modified some sources, but we did not modify Schema.pm or the DB.pm
I’m assuming I made a change too much, which is crucial to the REST interface.
I do hope you can forward my question to one of your Bugzilla experts to help me get this on track again.
Thank you in advance for any assistance you can provide.
Kind regards,
Ignace Danneels
Controls Design Engineer
TREMEC
Office: +32 (0) 50 977 766
From: Ignace Danneels
Sent: Wednesday, 29 March 2023 14:34
To: Dave Miller <justdave at bugzilla.org><mailto:justdave at bugzilla.org>; support-list at bugzilla.org<mailto:support-list at bugzilla.org>
Subject: RE: Bugzilla 5.0.6 Unable to log in after migration
Changes for MySQL 8:
(I had to dig into my knowledge of perl again. I haven’t used that for many years => switched to Python)
We have Bugzilla 5.0.3 running on MySQL 8 for quite some time now. Ever since IT decided to change to the latest version of MySQL! I think it’s up to ½ year running without issues now.
If we do, it’s because we forgot to update an SQL statement. So we fix that one, then simply continue.
The incompatibility towards MySQL 8 is basically caused by the fact that “groups” is now a reserved word.
So we changed the table name to ‘groeps’ . I’m from Belgium and ‘group’ in Dutch is ‘groep’. The plural is groepen, but we just changed one letter. So ‘groups’ has become ‘groeps’.
Then all I did was change all SQL statements to replace ‘groups’ with ‘groeps’. Only 80 replacements in 13 files.
When installing 5.0.6, I also had to modify the ABSTRACT_SCHEMA.
When running perl checksetup.pl, it tried to create the groeps table.
I found that there’s a table in the database bz_schema that contains the current layout of the database, so I changed the blob to replace all ‘groups’ by ‘groeps’ as well.
The next hurdle I encountered was in the sub _add_password_salt_separator
my $profiles
= $dbh->selectall_arrayref(
"SELECT userid, cryptpassword FROM profiles WHERE ("
. $dbh->sql_regexp("cryptpassword", "'^[^,]+{'")
. ")");
I’m quite sure it’s because the { in a regex indicates an amount of something to repeat. So I needed to escape that. It resulted in
my $profiles
= $dbh->selectall_arrayref(
"SELECT userid, cryptpassword FROM profiles WHERE ("
. $dbh->sql_regexp("cryptpassword", "'^[^,]+\\\\{'")
. ")");
That’s it. The perl checksetup.pl now ran without issues.
The site is running and I can see everything, but I can’t log in! Probably because the encrypted password in the table is an old one from the days before we used the LDAP. But I’m not sure.
I’ll look for the errorlog file. In the current server (Windows 2012), it is in the data folder. So I thought the new installation would also have it there.
Best regards,
Ing. Ignace Danneels
Controls Design Engineer
__________________________________________________________________________
TREMEC
Belgium Phone: +32 50 977766
From: Dave Miller <justdave at bugzilla.org<mailto:justdave at bugzilla.org>>
Sent: Wednesday, 29 March 2023 14:13
To: support-list at bugzilla.org<mailto:support-list at bugzilla.org>; Ignace Danneels <ignace.danneels at tremec.com<mailto:ignace.danneels at tremec.com>>
Subject: Re: Bugzilla 5.0.6 Unable to log in after migration
[CYBERSECURITY DESC] MENSAJE DE UN REMITENTE EXTERNO / MESSAGE FROM AN EXTERNAL SENDER
Toma precauciones extremas, no hagas click en los enlaces.
Take extreme precautions, do not click on unknown links.
Error 500 indicates that the webserver is encountering a problem which should be logged in the error log. That log exists somewhere, you'll need to find it. Since you're on Windows it might be in the system wide service logs in the control panel if it's not somewhere in Apache's directory structure.
Also, Bugzilla 5.0.6 is **not** compatible with MySQL 8. I'm curious to know what you did to make it work, though I suspect that might be why you're getting the error 500 (but we won't know for sure until you find the logs). My general suggestion is to use MariaDB instead (which is a drop in replacement for MySQL).
On March 29, 2023 7:55:01 AM EDT, Ignace Danneels <ignace.danneels at tremec.com<mailto:ignace.danneels at tremec.com>> wrote:
Hello,
I hope someone can help me.
Situation:
We currently have 5.0.3 running on a Windows server using a MYSQL 8 database. The site indicates Bugzilla doesn’t support MySQL 8, but the conversion was really easy.
We need to migrate to a new server and took the opportunity to install the latest version of Bugzilla (I downloaded the targ Bugzilla-5.0.6.tar.gz) and installed it on the new server.
I made a duplicate of the database (to ensure nothing gets corrupted while testing).
I installed StrawBerry Perl, then applied the same changes to ensure compatibility with MYSQL 8. I ran perl checksetup.pl and passed. The site is up and running, I can see the existing bugs and so one, but I can’t log in!
Then I read in the documentation about moving, so I copied the ‘old’ data folder to the new installation, but it results in the same error as below.
I looked at the profiles table, and my account does have a crypted password, but as the configuration was changed (ages ago) to use an LDAP, I doubt if the crypted password has anything to do with the LDAP password.
We use an ldap server, so I thought about copying ONLY the params.json file from the old server onto the new one (to transfer the settings), but then I get an error that it’s missing some resources and that’s it.
[cid:image003.png at 01D96245.EB9B9DD0]
It doesn’t indicate what the problem is, and there’s no errorlog on the server neither. So I’m just ‘floating in darkness’…
I would appreciate if someone could give me a helping hand.
I you could provide me with a profile record that has admin rights
Thank you (in advance) for any help you can offer me.
Best regards,
Ing. Ignace Danneels
Controls Design Engineer
TREMEC
TORQUE TRANSFER SOLUTIONS
Ignace.Danneels at tremec.com<mailto:Ignace.Danneels at tremec.com>
tremec.com<http://tremec.com/>
[cid:image001.jpg at 01D96244.287B07B0]
[cid:image002.png at 01D96244.287B07B0]
________________________________
Este correo electrónico es confidencial y está destinado exclusivamente para la(s) persona(s) a quien(es) se dirige. Queda estrictamente prohibida la distribución o copia del contenido de este correo. Si usted ha recibido este correo por error le suplicamos notificar inmediatamente a la persona que lo envió y borrarlo definitivamente de su sistema. Grupo Kuo, S.A.B de C.V. con domicilio en Paseo de los Tamarindos No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05110, Ciudad de México, México, podrá tratar los Datos Personales recabados vía correo electrónico con la finalidad de atender a las comunicaciones que nos haga llegar, darles seguimiento a las mismas, por motivos de seguridad y/o para cumplir con las obligaciones legales aplicables. Para más información, consulte nuestro Aviso de Privacidad en https://kuo.com.mx/avisos-privacidad. Este mensaje no constituye una aceptación, firma electrónica o un acuerdo vinculante, a menos que se especifique expresamente lo contrario en el mismo.
This e-mail is confidential and is intended only for the person(s) to whom it is addressed. Any distribution or copy hereof is strictly forbidden. If you have received this e-mail by error, we kindly ask you to notify the sender and to delete it immediately. Grupo Kuo, S.A.B. de C.V., having its address in Paseo de los Tamarindos, No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05120, Ciudad de México, Mexico, may process Personal Data collected via email in order to process the communications submitted by you, follow up on them, for security reasons and/or to fulfill applicable legal obligations. For more information, see our Privacy Notice at https://kuo.com.mx/avisos-privacidad. Nothing in this message is intended to constitute an acceptance, an electronic signature nor a binding agreement, unless a specific statement to the contrary is expressly included in this message.
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
________________________________
Este correo electrónico es confidencial y está destinado exclusivamente para la(s) persona(s) a quien(es) se dirige. Queda estrictamente prohibida la distribución o copia del contenido de este correo. Si usted ha recibido este correo por error le suplicamos notificar inmediatamente a la persona que lo envió y borrarlo definitivamente de su sistema. Grupo Kuo, S.A.B de C.V. con domicilio en Paseo de los Tamarindos No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05110, Ciudad de México, México, podrá tratar los Datos Personales recabados vía correo electrónico con la finalidad de atender a las comunicaciones que nos haga llegar, darles seguimiento a las mismas, por motivos de seguridad y/o para cumplir con las obligaciones legales aplicables. Para más información, consulte nuestro Aviso de Privacidad en https://kuo.com.mx/avisos-privacidad. Este mensaje no constituye una aceptación, firma electrónica o un acuerdo vinculante, a menos que se especifique expresamente lo contrario en el mismo.
This e-mail is confidential and is intended only for the person(s) to whom it is addressed. Any distribution or copy hereof is strictly forbidden. If you have received this e-mail by error, we kindly ask you to notify the sender and to delete it immediately. Grupo Kuo, S.A.B. de C.V., having its address in Paseo de los Tamarindos, No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05120, Ciudad de México, Mexico, may process Personal Data collected via email in order to process the communications submitted by you, follow up on them, for security reasons and/or to fulfill applicable legal obligations. For more information, see our Privacy Notice at https://kuo.com.mx/avisos-privacidad. Nothing in this message is intended to constitute an acceptance, an electronic signature nor a binding agreement, unless a specific statement to the contrary is expressly included in this message.
--
________________________________
Este correo electrónico es confidencial y está destinado exclusivamente para la(s) persona(s) a quien(es) se dirige. Queda estrictamente prohibida la distribución o copia del contenido de este correo. Si usted ha recibido este correo por error le suplicamos notificar inmediatamente a la persona que lo envió y borrarlo definitivamente de su sistema. Grupo Kuo, S.A.B de C.V. con domicilio en Paseo de los Tamarindos No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05110, Ciudad de México, México, podrá tratar los Datos Personales recabados vía correo electrónico con la finalidad de atender a las comunicaciones que nos haga llegar, darles seguimiento a las mismas, por motivos de seguridad y/o para cumplir con las obligaciones legales aplicables. Para más información, consulte nuestro Aviso de Privacidad en https://kuo.com.mx/avisos-privacidad. Este mensaje no constituye una aceptación, firma electrónica o un acuerdo vinculante, a menos que se especifique expresamente lo contrario en el mismo.
This e-mail is confidential and is intended only for the person(s) to whom it is addressed. Any distribution or copy hereof is strictly forbidden. If you have received this e-mail by error, we kindly ask you to notify the sender and to delete it immediately. Grupo Kuo, S.A.B. de C.V., having its address in Paseo de los Tamarindos, No. 400-B, piso 31, Bosques de las Lomas, Cuajimalpa de Morelos, C.P. 05120, Ciudad de México, Mexico, may process Personal Data collected via email in order to process the communications submitted by you, follow up on them, for security reasons and/or to fulfill applicable legal obligations. For more information, see our Privacy Notice at https://kuo.com.mx/avisos-privacidad. Nothing in this message is intended to constitute an acceptance, an electronic signature nor a binding agreement, unless a specific statement to the contrary is expressly included in this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 15853 bytes
Desc: image001.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 67962 bytes
Desc: image002.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0011.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 12110 bytes
Desc: image003.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0012.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 69922 bytes
Desc: image004.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0013.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 89656 bytes
Desc: image006.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0014.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 49413 bytes
Desc: image005.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0015.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 98549 bytes
Desc: image007.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0016.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.png
Type: image/png
Size: 95556 bytes
Desc: image008.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0017.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.png
Type: image/png
Size: 52617 bytes
Desc: image009.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0018.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image010.png
Type: image/png
Size: 54164 bytes
Desc: image010.png
URL: <http://lists.bugzilla.org/pipermail/support-list/attachments/20230901/47545f29/attachment-0019.png>
More information about the support-list
mailing list