From gerv at mozilla.org Tue Jan 2 10:50:08 2018 From: gerv at mozilla.org (Gervase Markham) Date: Tue, 2 Jan 2018 10:50:08 +0000 Subject: Bugzilla Meeting for January is tomorrow at 9pm UK time (1pm Pacific) Message-ID: The next monthly Bugzilla public meeting will be on Wednesday January 3rd, at 9pm UK time, 9pm UTC, 1pm Pacific. http://arewemeetingyet.com/London/2018-01-03/21:00/Bugzilla%20Meeting Agenda is here: https://wiki.mozilla.org/Bugzilla:Meetings Feel free to add stuff you want to talk about, and see you then :-) Gerv _______________________________________________ dev-apps-bugzilla mailing list dev-apps-bugzilla at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-apps-bugzilla From justdave at bugzilla.org Wed Jan 3 21:39:30 2018 From: justdave at bugzilla.org (Dave Miller) Date: Wed, 03 Jan 2018 16:39:30 -0500 Subject: Announcing Project Bugzilla Harmony In-Reply-To: References: Message-ID: <5A4D4D92.6060102@bugzilla.org> This got forwarded to me, I don't see that it ever made it to the mailing list, not sure where it ended up. But need to get these replies out there. * From: *Dylan Hardison > *Subject: **Re: Announcing Project Bugzilla Harmony* *Date: *December 1, 2017 at 10:50:29 EST *To: *developers at bugzilla.org > > On Dec 1, 2017, at 08:20, Gervase Markham > wrote: > > Hi Dylan, > > Thanks for this - it's good to see some progress on defining what we are > going to be doing :-) > > On 30/11/17 20:36, Dylan Hardison wrote: >> 1. Significant changes to BMO must *not* be required. >> 1.1. No requiring perl 5.14, though nothing prevents us from working >> *better* on modern perls. > > What are the current Perl requirements for stable, trunk and BMO? > > What is the objection to updating the version of Perl we require? > We declare a min version in our code, and this declaration changes the syntax to some degree. For instance, in trunk we have use 5.14.0, which also enables strict mode too (so use strict is not needed). BMO is targeting 5.10.1. It also runs under 5.16 quite well, because 5.16 is used by version-control-tools, and it also runs under 5.24 and 5.26 just fine. But we can't rely on features from those until BMO production is migrated. In about six months I suspect BMO will be running on alpine linux and perl 5.26. I very much want to use only the supported (by the perl community) versions of perl, which is just "the last two versions", but the reality is that I can't upgrade overnight. So for harmony, I'd like to have the minimum version be 5.10.1. I plan on porting https://github.com/mozilla-bteam/bmo-systems to upstream bugzilla as well, and >> 1.2. The usernames != login patch has to be backed out (someone >> started this, and I've asked gerv to review it on GitHub) > > Remind us all why this needs to happen? > > Have we shipped this code to anyone? People running the master branch are definitely using it. The problems are a few: it's possible for the DB to be out of sync with the parameter, and if you're registering users when out of sync "strange" things could happen. For BMO the plan is to merely lax the requirement on the login_name field, after adding (multiple) email addresses to the account. This is similar to what many other platforms do. https://bugzilla.mozilla.org/show_bug.cgi?id=1372631 > >> 2. We are going to follow https://rfc.zeromq.org/spec:42/C4/ >> with the understanding that >> "Platform issue tracker" refers to bugzilla.mozilla.org >> > > What is your goal here? What about our current development model is > broken and needs fixing? Are there specific bits of this rather long > document which you are particularly concerned we adopt? > >> 3. All this work will happen to the "harmony" branch, and we'll >> continue to not touch 'master' to not break anyone that happens to be >> using it. > > I assume "harmony" will be a branch in > http://github.com/bugzilla/bugzilla? > > > Will "harmony" start from master, stable, or from the BMO codebase? > > I was rather expecting to start with the BMO code and work back towards > Bugzilla compatibility, rather than starting with master and working > towards BMO compatibility. The former certainly seems like it would be > easier to do... > >> We are not in the business of packaging, there are too many >> combinations to worry about all of them. >> People are welcome to help with packaging Bugzilla 6 / Harmony on >> anything, and nothing drastic will change >> there. But as for releasable products, there are two channels that we >> must support: >> >> 1. A runnable docker container, based on Alpine linux > > I have no particular objection, but why Alpine Linux? What is that? A very small linux distro and the de-facto base image used by high-quality docker containers. > >> 2. A checkout from a tagged commit from GitHub. >> >> The release process for Bugzilla Harmony will amount to tagging a >> commit (from the Harmony branch, more on that later). > > By "release process", do you mean "the process which result in the thing > we call e.g. Bugzilla 6.0"? Or are you talking about some testable > milestones we might ship along the way? > Only the former. I'm going for continuous releases. I'm never going to have time for the other release process. >> The release notes should be exactly the commit messages. Part of >> reviewing a PR is making sure the commit messages tell the >> appropriate story. > > I don't think this particular point is going to fly. It means you can't > really tell any story which spans multiple commits, and it means you > won't be picking out particular features as worth promoting. Doing > things this way will just result in pretty useless release notes. I > understand and agree with your desire to reduce the work involved in > shipping a release, but I think this is an over-economy. You can ignore a subset of commits -- I use this to build the announcements for BMO for the last 4 years. I also think with a continuous release process it is less interesting. We can say "pushing built images is not a release", but these non-releases are the only thing I'm committed to doing or supporting. > >> Commits / Reviews / Pull Requests: >> >> Read the C4 document, that is what contributing to Bugzilla's harmony >> branch should be like. > > That is a somewhat intimidating statement; see my question above. We > have been discussing a significant change to _what_ we are building; a > significant change to _how_ we are building it is somewhat of a bolt > from the blue. I would very much appreciate it if you were to explain > all this a bit more. > The existing process has not worked, it's burdensome. The C4 process is effective for a variety of successful projects. That document gives clear expectations for a lot of situations and encourages patches to actually land. I think that's something important. >> 4. Revert the REST API in favor of the (older) system that BMO is >> still using. > > Why is that necessary? Cannot a new, improved API and the old one live > in parallel? > (explained upthread) >> 5. Ensure a database created by 4.2, 5.0, or the master branch can be >> "upgraded" to from the harmony branch. > > I think you mean the other way around :-) yeah. > >> 10. Port GitHubAuth (this has already been done!) > > Surely the harmony branch needs eventually to support all BMO's > extensions? > Absolutely! The reason harmony is starting from 5.0 is that the other major stake-holder is running (or will be shortly) 5.0 (RedHat). I hope they open up their fork's repo, and I will continue to hope for that. I'm already going to be asking them if they can back-out on multiple bug aliases. Handling the conflicts between BMO and 5.0 is a burden I am able to bear, so I won't ask my RedHat friends to shoulder it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From justdave at bugzilla.org Wed Jan 3 21:48:18 2018 From: justdave at bugzilla.org (Dave Miller) Date: Wed, 03 Jan 2018 16:48:18 -0500 Subject: Announcing Project Bugzilla Harmony In-Reply-To: References: Message-ID: <5A4D4FA2.7050905@bugzilla.org> Here's another one I found stuck in the moderator queue when I went looking to see what happened to Dylan's other post: Subject: Re: Announcing Project Bugzilla Harmony From: Dylan Hardison > Date: Fri, 1 Dec 2017 10:23:57 -0500 To: developers at bugzilla.org > On Dec 1, 2017, at 09:56, Denis Roy wrote: > > This is great news! I will follow along and contribute however I can, > with the very few cycles that I have > > Some comments below: > > > On 12/01/2017 08:20 AM, Gervase Markham wrote: >>> Commits / Reviews / Pull Requests: >>> >>> Read the C4 document, that is what contributing to Bugzilla's harmony branch should be like. >> That is a somewhat intimidating statement; see my question above. > +1 Not sure I follow. >>> 4. Revert the REST API in favor of the (older) system that BMO is still using. >> Why is that necessary? Cannot a new, improved API and the old one live >> in parallel? The "new" one is just a code migration from the existing one. It is "mostly" API compatible, but we couldn't use it on BMO as there were too many differences. It is required for harmony to leave the REST API that is used on BMO alone. To support the work Kohei is doing on the UX side, I will add an /api/ or /rest2/ endpoint and provide new features there -- completely separate from the JSONRPC/XMLRPC/REST framework that exists now. It is really easy to just drop in a new .cgi and support REST directly, and leave the (BMO/5.0/etc) REST system in place. All work on APIs follows this guideline from the C4 process: 2.6. Evolution of Public Contracts ? All Public Contracts (APIs or protocols) SHALL be documented. ? All Public Contracts SHOULD have space for extensibility and experimentation. ? A patch that modifies a stable Public Contract SHOULD not break existing applications unless there is overriding consensus on the value of doing this. ? A patch that introduces new features SHOULD do so using new names (a new contract). ? New contracts SHOULD be marked as "draft" until they are stable and used by real users. ? Old contracts SHOULD be deprecated in a systematic fashion by marking them as "deprecated" and replacing them with new contracts as needed. ? When sufficient time has passed, old deprecated contracts SHOULD be removed. ? Old names SHALL NOT be reused by new contracts. -------------- next part -------------- An HTML attachment was scrubbed... URL: From justdave at bugzilla.org Fri Jan 5 06:01:58 2018 From: justdave at bugzilla.org (Dave Miller) Date: Fri, 05 Jan 2018 01:01:58 -0500 Subject: Looking back at Bugzilla and BMO in 2017 Message-ID: <5A4F14D5.4040305@bugzilla.org> Reposted from Dylan's tumblr with permission. -----8<----- * Looking back at Bugzilla and BMO in 2017 Looking Back Recently in the Bugzilla Project meeting , Gerv informed us that he would be resigning, and it was pretty clear that my lack of technical leadership was the cause. While I am sad to see Gerv go, it did make me realize I need to write more about the things I do. As is evident in this post, all of the things I?ve accomplished have been related to the BMO codebase and not upstream Bugzilla ? which is why upstream must be rebased on BMO. See Bug 1427884 for one of the blockers to this. Accessibility Changes In 2017, we made over a dozen a11y changes, and I?ve heard from a well-known developer that using BMO with a screen reader is far superior to other bugzillas. :-) Infrastructure Changes BMO is quite happy to use carton to manage its perl dependencies, and Docker handle its system-level dependencies. We?re quite close to being able to run on Kubernetes. While the code is currently turned off in production, we also feature a very advanced query translator that allows the use of ElasticSearch to index all bugs and comments. Performance Changes I sort of wanted to turn each of these into a separate blog post, but I never got time for that ? and I?m even more excited about writing about future work. But rather than just let them hide away in bugs, I thought I?d at least list them and give a short summary. February o Bug 1336958 - HTML::Tree requires manual memory management or it leaks memory. I discovered this while looking at some unrelated code. o Bug 1335233 - I noticed that the job queue runner wasn?t calling the end-of-request cleanup code, and a result it was also leaking memory. March o Bug 1345181 - make html_quote() about five times faster. o Bug 1347570 - make it so apache in the dev/test environments didn?t need to restart after every request (by enforcing a minimum memory limit) o Bug 1350466 - switched JSON serialization to JSON::XS, which is nearly 1000 times faster. o Bug 1350467 - caused more modules (those provided by optional features) to be preloaded at apache startup. o Bug 1351695 - Pre-load ?.htaccess? files and allow apache to ignore them April o Bug 1355127 - rewrote a template that is in a tight loop to Perl. o Bug 1355134 - fetch all group at once, rather than row-at-a-time. o Bug 1355137 - Cache objects that represent bug fields. o Bug 1355142 - Instead of using a regular expression to ?trick? Perl?s string tainting system, use a module to directly flip the ?taint? bit. This was hundreds of times faster. o Bug 1352264 - Compile all templates and store them in memory. This actually saved both CPU time and RAM, because the memory used by templates is shared by all workers on a given node. May o Bug 1362151 - Cache bzapi configuration API, making ?bz export? commands (on developer machines) faster by 2-5 seconds. o Bug 1352907 - Rewrite the Bugzilla extension loading system. The previous one was incredibly inefficient. June o Bug 1355169 - Mentored intern to implement token-bucket based rate limiting. Not strictly a performance thing but it reduced API abuse. December o Bug 1426963 - Use a hash lookup to determine group membership, rather than searching an unsorted list. Bug 1427230 Developer Experience Changes My favorite communities optimize for fun. Frequently fun means being able to get things done. So in 2017 I did the following: o Made a vagrant development environment setup that closely mapped to BMO production. ** I tested installing it on various machines ? Linux, OSX, Windows ** I wrote a README explaining how to use it. ** This dev environment has been tested by people with little or no experience with Bugzilla development. o I changed to a pull-request based workflow. We use Bugzilla to track bugs and tasks, but not do code review. o I made it so the entire test suite could run against pull requests. This isn?t trivial, you have to work a bit harder to build docker images and run them without having any dockerhub credentials. (Pull requests don?t get any dockerhub credentials, I have to say to make sure my friend ulfr doesn?t have a heart attack) o I made sure that I understood how to use Atom and Visual Studio Code. I actually rather like the later now ? and more importantly it is easy to help out new-comers with these editors. o I adopted Perl::Critic for code linting and Perl::Tidy for code-formatting, using the PBP ruleset for the later. I also made it a point to not make code style a part of code review ? let the machine do that. Numbers In the last year, we had almost 500 commits to the BMO repo, from 20 different people. Some people were new, and some were returning contributors (such as Sebastin Santy). -----8<----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dylan at mozilla.com Mon Jan 8 00:21:23 2018 From: dylan at mozilla.com (Dylan Hardison) Date: Sun, 7 Jan 2018 19:21:23 -0500 Subject: Project updates Message-ID: <3634D0C8-EFB8-4894-81D0-693F6F8C6F5A@mozilla.com> I've revised https://wiki.mozilla.org/Bugzilla:Roadmap to more clearly articulate what needs to be done for a release. Let's discuss how we can make sustainable and frequent releases. Also, I motion that since we can't as-yet point bugzilla.org to the bugzilla.github.io site Gerv setup, we update the bugzilla.org to say that it is not "current" and direct people to bugzilla.github.io until such time that we are able to update bugzilla.org. Alternatively, I offer my bugzilla.ninja domain for this purposes.