We need to clean up bug_email and move it from contrib to the main code.

Besides the feature presently possible. I'd like to see: a) Allow product/component selection based on the "To:" address. (We have only one product and a few compontents a support-unix@ and support-win@ is easier than bugs@ + @product one + @component unix b) Allow the automatical addition of the user to the database (it the account doesn't exist yet). This would provide a simpler access for in-house support and would be like Debian's bug tracking system (minus "reportbug") c) It should be possible to separate mailserver and webserver. .procmailrc is run on the mailserver and ConnectToDatabase(); has to be run on the bugzilla-webserver.

This bug is listed on the 2.18 goals list. This is now a meta bug for all issues required to get bug_email into a condition that we can officially support it. If anyone knows of any other bugs related to bug_email which didn't get set as dependencies here, please add them.

The fix for this bug involves ignoring whats there entirely, and instead moving the backend for bug creation/modification out of post_bug.cgi, and into a backend module which the procmail script shares.

This is a very alpha version of my proposal of a new bugzilla mail interface to create bugs from mail- or other input. It is independant from the Bugzilla server (needs not to run on the same computer) and generates a http-request to post_bug.cgi with the input data. The script is not limited to mail input. See README in the attachment tarball for few further notes.

The approach of running the mail parser on another machine and communicating by POST is appealing, though I am not sure how you are handling authentication. Before you go too far with this, I am not sure if the existing form variables are the best way to seperate the front end and back end. Should this be something done by POSTing an XML description of the bug?

Since this hasn't been touched since September, I'd like to take a stab at owning it for a couple of weeks and see what I can do with it. I planned on updating the bug email documentation anyway, and in doing so discovered how unmaintainable existing inbound mail handling was.

note all the dependencies as well :)

Well, I am still here to help with a new version of the mail interface :) I have the new version running in production now. Unfortunately it can only create bugs and append comments and there is still much room for improvement. Please drop me a note if you want me to attach the latest tarball ;-) I fully agree that it would be a much better way to send xml description of the bug and not to form a post request filling formular data. But in 2.16, there is no such XML backend. Is it here in 2.17.x.? This is the reason why there is no bug manipulation in my mail interface yet. For authentication, the bugzilla password must send as a tag in the mail that creates a bug. Thats bad, of course, but not as bad as having no authentication. A future extension could use PGP and signed mails.

No longer a 2.18 blocker. Gerv

I recently upgraded my bugzilla install at work to Bugzilla 2.17.4. We immediately noticed that bug_email and bugzilla_email_append were broken by the integration of the venerable processmail.pl into the main code. We fixed bug_email and bugzilla_email_append to work with the corified processmail functions. I've attached the patched versions of the code. Hopefully somebody with CVS acess can test them and put them in contrib, since it seems that Gerv is abandoning this bug once again.

Brian, can I ask you to file a new bug on updating them to the trunk and attaching cvs diffs of the files? This bug is a tracker, and shouldn't be used to house any specific development.

Brian: Gerv doesn't own this bug, so he's not abandoning it. Matt obviously abandoned it a long time ago though, so changing the owner to reflect reality. There's already FIXED bugs for the processmail changes. It was fixed in 2.17.5. (see bug 192512)

This really needs a full rewrite.... What I'd like to see is something that does the following... 1) Accepts mail in text/plain or multipart/alternative with a text/plain section. 2) Configurable to permit new bugs to be assigned to products/components by either a) Mapping the email address to a command-line for bug_email.pl (e.g. BugzillaBugs: "| bug_email.pl Bugzilla Bugzilla-General" or BugzillaBugs: "| bug_email.pl Bugzilla --default Bugzilla-General" ) b) Parsing the subject line Subject: [Bugzilla-General] Integrate bug_email into the main code c) Parsing the body Component: Bugzilla-General 3) Permit mailed-in bugs to be assigned to a default owner for classification a) Either all incoming bugs or only those incompletely specified by parsing b) It should also be possible to have ambiguous/incomplete mails forwarded by email to a default owner for repair. The default owner should be able to forward the fix message back to bug_email and have it processed as an action by the original author rather than the person who fixes the message. 4) Permit bugs to be updated by email following mailto: links in bugmail (e.g. Subject: [bug 94850] ....) a) Comments should simply be appended. b) Alternative HTML representations should be ignored. c) Attachments should be attached i) Needs protection against silly attachments like Lotus Notes PCX files, probably by requiring that commands be placed in the text of the mail. (e.g. ATTACH: filename ) - otherwise the file is ignored. d) Commands should include REASSIGN: user-email RESOLVE: FIXED ACCEPT: DUPLICATE: number 5) All mail-in actions should be confirmed by email automatically 6) In order to make this viable, outbound bugmail should be easily customizable to indicate the group permissions of a bug. (So users don't accidentally mail-in would-be private comments on public bugs) 7) Private comment mechanisms need to be relatively goof-proof.

I think the points mentioned here can be fullfilled with my proposal for the mail interface 2 in comment #4. I am willing to help if you agree.

I think some early decisions are needed on the following questions.... 1) How should the script interact with the back-end? My suspicion is that the right thing to do is to POST the entire SMTP message and let all the parsing happen on the server where the database is accessible. This would minimize issued of syncing form variables and solve many problems with permissions. 2) Should we always take the word of the user in the email From: field about the user's identity or is more validation needed?

I think a user preference for how mail from them is handled would be in order. When a new user signs up, the system would default to not allowing mail from them. In their preferences is three (or more?) choices for how to deal with mail received with their address on the From: line: 1) don't allow mail from me (default) 2) allow mail from me and trust the From: line 3) allow mail from me and require it to be GPG-signed with this key: (textbox) Another possibility is temporarily stashing received email somewhere and issuing a token for it, which is mailed back to the user to confirm (this is what most mailing list systems do)

(In reply to comment #16) > I think the points mentioned here can be fullfilled with my proposal for the > mail interface 2 in comment #4. I am willing to help if you agree. We'd love the help, though I think some further refinement is needed on the proposal in comment 4. We certainly want to be able to create and alter bugs by email. These changes must be handled under the bugzilla account appropriate to the user. Using a password in the email would be a pretty poor security approach. The suggestions justdave made in comment 18 are a good start, though I'd like to come up with a good way to validate users who would never figure out GPG. One alternative would be to permit justdave's option 2 require that the mail come from certain hosts. Anyone have a better idea?? So far, justdave and I have come up with a whole array of partial solutions. A real solution would be handy :-)

I would like to add a suggestion to this: Any implementation that handles this should have two parts: Part 1: Parse e-mail (in whatever form) into a hash containing all the information for creating/updating the bug (I'm more concerned with creating them myself), call Part 2 with this hash and handle results. Part 2: HTTP/hash API: A funtion callable from anywhere that: Gets a hash with information for creating/updating the bug. Takes that hash, forms the HTTP requests, sends them off, gets a result, parses a result, puts together a new hash ({bug_number}=>..., etc) and returns that hash. This way, when you have anyone who wants to, say, migrate a different bug database to bugzilla, automatically add 80 bugs, etc, they don't have to send 80 e-mails, they can just write a script around this API, pass in a hash, and deal with the results. It shouldn't be too much trouble to seperate these functionality, and would make extending it much easier.

That sounds like Bug.pm, if only it could write out changes. :)

Rumor has it that someone is working on this... anyone?

Sorry abbout the essay, but here is what I see for this: A lot of the bug manipulation routines need to be put in a module that makes it easier for other parts of the program, like mail-in, to use. Now, I think it should be considered beyond the scope of this bug to go delving into Bug.pm and similar places to make everything go along with this. It needs to be done, but I think a good shove in the right direction would be to create duplicate functionality in a more generic library on an as-needed basis, then work on migrating the old code over in another bug. I'm a really big fan of trying to get these patches in in small steps, so I'd like to keep the capabilities of an initial mail-in system to a minimum, while leaving it wide open for expansion later. A simple version of inbound mail would have a set of options defined line-by-line, like: Command: Search Product: Bugzilla Component: Bugzilla-General ...and so forth. Optionally, a Command could be listed on the subject line, along with a default option. In Search's case, I would assume short_desc would be the default field, so an email message coming in that has: Subject: Search bug_email ...would search the Summary (short_desc) for "bug_email" Various commands should be handled by their own modules. It should probably just include anything that's in Bugzilla/Mail/Inbound. Function refs for each command are kept in a hash, keyed by command name. We pass the options and values to it as a hash ref. Functions should also have access to all of the MIME portions of a message, along with any uuencoded data. Inbound messages have to pass through some authorization tests, including host mask (authorize any host on 192.168.0.0/16, if all relays occurred there), confirmation key (unauthorized mail is sent back to its origin with a key string inside, saying "please just respond to authorize"). Something involving GPG signatures or a persistent key would be nice in the future. Help should allow you to request a template message for any command, with all possible options listed and brief explanations so a user may fill in the blanks. To start with, I would like to include very simple Help and New-Bug commands. Internationalization kind of scares me, since the commands and options will probably want to live in a template somewhere, then be fed back into the code Anyhow, due to the fact that I need this, I'm willing to get rolling on it, unless I hear objections.

Small steps, minimal functionality, separate modul that gradually takes over the work of spagetti --> this surely is the most reasonable (and by-the-book) path to go - couldn't agree more with Erik. Regarding the interface though, this should be done very carefully. Utmost simplicity is what is wanted of posting a new bug. At least some advanced capabilities (like in quicksearch.html) are wanted from searching through bugs. Other commands like help, user-mgmt, product-mgmt are far less important and could probably wait for interface to stabilize a bit, or they could exist in some most basic form. Internationalization could probably also be postponed and added (or "refactored") later. Of course, my oppinions are based on needs and experience of mine, my colleagues and bugzilla users on the same system. From this experience, e-mail interface is most badly needed when I finish session with end-user testers and want to enter 30 bugs at once. Bugzilla via html interrupts me too much when I write all the data I could gather and try to comment as much as possible. So I write all in advance in text editor, and then *slowly* post problem descriptions and attachments. This process is very tedious and leeds to situation that sometimes people post bugs "in chunks": one bug that containes descriptions of 10 associated bugs. Sorry, I also got a little lengthy, just wanted to help. Thanks, Erik.

I like the idea of (optional) RFC2822-style headers within the body of the message. Everyone's used to it already from email, and it's also the way Debian's mail interface works (which seems to be the most well-liked of the bug trackers with inbound email support currently [at least by people who like to work via email - everyone else hates it]).

OK, I'll make this bug my burden. Not today, but soon, because it's something I need. I want to write a brand-new modular system for this, and it's probably going to require a *lot* of params, so I'm putting bug 46296 (multi-panel editparams) on as a dependency. Otherwise the process of setting it up will just be ridiculous.

Bugzilla 2.20 feature set is now frozen as of 15 Sept 2004. Anything flagged enhancement that hasn't already landed is being pushed out. If this bug is otherwise ready to land, we'll handle it on a case-by-case basis, please set the blocking2.20 flag to '?' if you think it qualifies.

Adding a few keywords to the summary to make it easier to find.

*** Bug 304390 has been marked as a duplicate of this bug. ***

The trunk is now frozen to prepare Bugzilla 2.22. Only bug fixes are accepted, no enhancement bugs. As this bug has a pretty low activity (especially from the assignee), it's retargetted to ---. If you want to work on it and you think you can have it fixed for 2.24, please retarget it accordingly (to 2.24).

Okay, in present time, the real blocker of this bug is bug 122922 -- we need to have methods in Bug.pm of creating/modifying bugs before we can integrate the email interface into the main code. We shouldn't get into too much re-writing of the system; let's just make it work. This bug will just be for the final cvs add/cvs remove of the files out of contrib/.

This bug is in the new roadmap for 2.24.

(In reply to comment #32) > This bug is in the new roadmap for 2.24. Per the discussions in the meeting, and the roadmap document, I said I'd take this.

I'll take this. I have it entirely done, for 2.22.

Okay, here's the inbound email interface that I wrote for Akamai, for 2.22. The script itself is very nice. The modifications to attachment.cgi, process_bug.cgi, and post_bug.cgi are all sort of a hack, although they work very nicely for 2.22. For upstream I'd like to do something a little smoother.

Just so everybody knows the patch I attached is basically a port of bug_email's interface to entirely new code. That is, the code is all mine, but it still uses the same sort of email format as bug_email did, mostly because the email format made sense. It requires users to specify the fields at the top of the email, but the field names have to start with @. That's the only reliable way to tell if somebody's sent us an email with just a comment or not. It also allows an unlimited amount of whitespace at the top of the email. The script does allow more fields to be specified than bug_email did. It allows anything that post_bug or process_bug allows, in terms of field names. It does handle multipart/alternative mails just fine, although you can't include an attachment with a multipart/alternative mail (but only because of a bug in Email::MIME::Attachment::Stripper). The script *does* require a lot of the Email:: modules. If you're on RHEL, the ones that aren't available at Dag I've used cpan2rpm to make RHEL4 modules of: http://landfill.bugzilla.org/perl-email/ Validating the "From" address is something we should do in a future bug. Adding the ability to do things other than update bugs is something we could possibly do in a future bug.

Okay, and here's the basic framework for Bugzilla 3.0. This is just the email_in.pl script itself and the changes to Bugzilla required for it to compile. We're going to split this into three bugs: 1) The ability to create bugs 2) The ability to post bugs. 3) The ability to add attachments. Basically, I'll just be stripping out parts of this script and putting them in separate patches. Each patch will also require other changes to parts of Bugzilla, in order to work, so each patch will also include additional code.

(In reply to comment #35) > Created an attachment (id=236290) [edit] > Inbound Email Interface for Bugzilla 2.22 > Okay, here's the inbound email interface that I wrote for Akamai, for 2.22. > The script itself is very nice. The modifications to attachment.cgi, > process_bug.cgi, and post_bug.cgi are all sort of a hack, although they work > very nicely for 2.22. For upstream I'd like to do something a little smoother. Can you briefly explain how this patch works and how to install it. Many thanks JK

(In reply to comment #38) > Can you briefly explain how this patch works and how to install it. Sorry, the patch itself is unsupported. I'm sure you can find some tutorials online about how to apply a patch using the standard "patch" utility, though. If you want to know how the email interface works, you can do "perldoc email_in.pl" after you apply the patch. Make sure that you don't ask any more questions about it in this bug, though, since it's unsupported. It's possible you could ask questions about it on the support list, if you want to know any more about it.

Bugzilla 3.2 is now frozen. Only enhancements blocking 3.2 or specifically approved for 3.2 may be checked in to the 3.2 branch. If you would like to nominate your enhancement for Bugzilla 3.2, set "blocking3.2" tp "?", and either the target milestone will be changed back, or the blocking3.2 flag will be granted, if we will accept this enhancement for Bugzilla 3.2.

I would say that this is in fact fixed since Bugzilla 3.0 and email_in.pl. We just need to add attachment support, but that's a separate bug.