This is a tracking bug to track rolling out http://github.com/LegNeato/bugzilla-amqp on bmo. I see these as needing to happen before the extension is put into production: 1. There are some features that need to be added first (like, uh, security) 2. After that, because there is a fair amount of code (as far as Bugzilla extensions go), it will likely need to go through a security review 3. Performance testing needs to happen so that it doesn't bring down bmo inadvertently 4. The server running Mozilla Pulse needs to get beefier and the traffic expectations with IT have to be revisited (I promised them it was a prototype after all...) 5. An actual QA plan and testing

I think the changes for #1 are good enough (tm). * I added support for comments (if you patch Bugzilla). If you don't patch bugzilla, the extension will still work but will be blind to comments. * In http://github.com/LegNeato/bugzilla-amqp/commit/9e6ab808a354dc2fd8a89e1cb3fbb40131d4d791 I added some simple security. It will mean that tools are blind to anything restricted/secure, but I believe it is enough to roll-out the extension w/o security implications.

Updating as I changed the name...I plan to add more security options this week.

This is the general plan I am working with to get this rolled out: 1. Use metric's bugzilla activity tool to simulate load on pulse.mozilla.org 2. Backport bug 587793 if a new 3.6 release hasn't come out with the fix 3. Do NOT backport bug 590334 ** This means that pulse will be blind to comments ** Reduces load on pulse.mozilla.org ** Reduces risk, can later backport if not having comment messages is detrimental 4. Have the bugzilla-push extension publish to a local broker (w/o SSL) ** This keeps bmo responsive for users ** If pulse.mozilla.org goes down messages will be spooled locally and not lost 5. Use the rabbitmq-shovel plugin to shovel sets of messages from the local broker to pulse.mozilla.org (with SSL) Outstanding questions: 1. Bugzilla's architecture...would we have to have a local rabbit install on multiple machines? 2. If pulse.mozilla.org goes down, what resources (disk, memory, cpu) will be used by the local spooling?

(In reply to comment #3) > 2. Backport bug 587793 if a new 3.6 release hasn't come out with the fix Bugzilla 3.6.3 (what bmo is running) already has that fix, afaik.

Not going to make the BMO 4.0 roll out. We will work on this post 4.0. dkl

Found this bug through bug 692173. Is this still being worked on?

(In reply to James Long (:jlongster) from comment #7) > Found this bug through bug 692173. Is this still being worked on? yes, this is still on my list of things to address.

Any chance of this becoming bidirectional? to become another API for creating bugs etc.? So one would have to use only one middleware instead of bugzilla-push together with XML-RPC ?

(In reply to Jürgen Weber from comment #9) > Any chance of this becoming bidirectional? to become another API for > creating bugs etc.? > So one would have to use only one middleware instead of bugzilla-push > together with XML-RPC ? sorry, no, this will be a uni-directional interface only.

Any updates on potential rollout of this?

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #11) > Any updates on potential rollout of this? we're looking at bringing this partially into production soon (within a month). initially we're limiting the scope of integration with direct pushes to other systems on a very limited subset of messages, to gauge the impact this has on load, etc.

Great! Can you elaborate on what messages will be included? (Or, if it's easier, what will be dropped?) It seems that for bugzilla there's only a few message types, and I hope that bug created/bug changed (flag, comment, etc.)/attachment created/attachment changed are all included :)

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #13) > Great! Can you elaborate on what messages will be included? (Or, if it's > easier, what will be dropped?) oh, sorry i wasn't very clear in my response. initially none of the messages will be sent to pulse :( part of the refactoring was to make the bugzilla-push extension a generic "push to another system" framework, and the initial deployment will be to push updates to selected bugs directly to a b2g partner. > I hope that bug created/bug changed (flag, comment, > etc.)/attachment created/attachment changed are all included :) that list covers all changes to a bug, so i don't consider that limited ;)

(In reply to Byron Jones ‹:glob› from comment #14) > (In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #13) > > Great! Can you elaborate on what messages will be included? (Or, if it's > > easier, what will be dropped?) > > oh, sorry i wasn't very clear in my response. initially none of the > messages will be sent to pulse :( > > part of the refactoring was to make the bugzilla-push extension a generic > "push to another system" framework, and the initial deployment will be to > push updates to selected bugs directly to a b2g partner. That's unfortunate.. is the potential load really worse than various tools that are querying bugzilla directly? > > I hope that bug created/bug changed (flag, comment, > > etc.)/attachment created/attachment changed are all included :) > > that list covers all changes to a bug, so i don't consider that limited ;) Well, from what I saw, the push extension also included things like changes to groups etc. which are largely uninteresting...

> That's unfortunate.. is the potential load really worse than various tools that > are querying bugzilla directly? my guess is it's cheaper to push, and even if it isn't, we'll be in a position to spread the load across multiple systems, something which we can't do in response to high amounts of polling. the plan is to gather data around that question. > Well, from what I saw, the push extension also included things like changes to > groups etc. which are largely uninteresting... agreed; they have been dropped from being published. Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.0/ modified extensions/Push/Config.pm modified extensions/Push/Extension.pm added extensions/Push/bin added extensions/Push/lib added extensions/Push/template added extensions/Push/web added extensions/Push/bin/bugzilla-pushd.pl added extensions/Push/lib/Admin.pm added extensions/Push/lib/BacklogMessage.pm added extensions/Push/lib/BacklogQueue.pm added extensions/Push/lib/Backoff.pm added extensions/Push/lib/Config.pm added extensions/Push/lib/Connector added extensions/Push/lib/Connector.disabled added extensions/Push/lib/Connectors.pm added extensions/Push/lib/Constants.pm added extensions/Push/lib/Daemon.pm added extensions/Push/lib/Log.pm added extensions/Push/lib/LogEntry.pm added extensions/Push/lib/Logger.pm added extensions/Push/lib/Message.pm added extensions/Push/lib/Option.pm added extensions/Push/lib/Push.pm added extensions/Push/lib/Queue.pm added extensions/Push/lib/Serialise.pm added extensions/Push/lib/Util.pm added extensions/Push/lib/Connector/AMQP.pm added extensions/Push/lib/Connector/Base.pm added extensions/Push/lib/Connector/File.pm added extensions/Push/lib/Connector/TCL.pm added extensions/Push/lib/Connector.disabled/ServiceNow.pm added extensions/Push/template/en added extensions/Push/template/en/default added extensions/Push/template/en/default/hook added extensions/Push/template/en/default/pages added extensions/Push/template/en/default/setup added extensions/Push/template/en/default/hook/admin added extensions/Push/template/en/default/hook/global added extensions/Push/template/en/default/hook/admin/admin-end_links_right.html.tmpl added extensions/Push/template/en/default/hook/global/code-error-errors.html.tmpl added extensions/Push/template/en/default/hook/global/messages-messages.html.tmpl added extensions/Push/template/en/default/hook/global/user-error-errors.html.tmpl added extensions/Push/template/en/default/pages/push_config.html.tmpl added extensions/Push/template/en/default/pages/push_log.html.tmpl added extensions/Push/template/en/default/pages/push_queues.html.tmpl added extensions/Push/template/en/default/pages/push_queues_view.html.tmpl added extensions/Push/template/en/default/setup/strings.txt.pl added extensions/Push/web/admin.css added extensions/Push/web/admin.js Committed revision 8381. Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.2/ added extensions/Push added extensions/Push/Config.pm added extensions/Push/Extension.pm added extensions/Push/bin added extensions/Push/lib added extensions/Push/template added extensions/Push/web added extensions/Push/bin/bugzilla-pushd.pl added extensions/Push/lib/Admin.pm added extensions/Push/lib/BacklogMessage.pm added extensions/Push/lib/BacklogQueue.pm added extensions/Push/lib/Backoff.pm added extensions/Push/lib/Config.pm added extensions/Push/lib/Connector added extensions/Push/lib/Connector.disabled added extensions/Push/lib/Connectors.pm added extensions/Push/lib/Constants.pm added extensions/Push/lib/Daemon.pm added extensions/Push/lib/Log.pm added extensions/Push/lib/LogEntry.pm added extensions/Push/lib/Logger.pm added extensions/Push/lib/Message.pm added extensions/Push/lib/Option.pm added extensions/Push/lib/Push.pm added extensions/Push/lib/Queue.pm added extensions/Push/lib/Serialise.pm added extensions/Push/lib/Util.pm added extensions/Push/lib/Connector/AMQP.pm added extensions/Push/lib/Connector/Base.pm added extensions/Push/lib/Connector/File.pm added extensions/Push/lib/Connector/TCL.pm added extensions/Push/lib/Connector.disabled/ServiceNow.pm added extensions/Push/template/en added extensions/Push/template/en/default added extensions/Push/template/en/default/hook added extensions/Push/template/en/default/pages added extensions/Push/template/en/default/setup added extensions/Push/template/en/default/hook/admin added extensions/Push/template/en/default/hook/global added extensions/Push/template/en/default/hook/admin/admin-end_links_right.html.tmpl added extensions/Push/template/en/default/hook/global/code-error-errors.html.tmpl added extensions/Push/template/en/default/hook/global/messages-messages.html.tmpl added extensions/Push/template/en/default/hook/global/user-error-errors.html.tmpl added extensions/Push/template/en/default/pages/push_config.html.tmpl added extensions/Push/template/en/default/pages/push_log.html.tmpl added extensions/Push/template/en/default/pages/push_queues.html.tmpl added extensions/Push/template/en/default/pages/push_queues_view.html.tmpl added extensions/Push/template/en/default/setup/strings.txt.pl added extensions/Push/web/admin.css added extensions/Push/web/admin.js Committed revision 8403.

Excited to see this roll out! Where's the github pull request :-D? Or, it looks like it changed enough to be a different project now?