I use mutt as my email client and it has this cool feature where it can automatically arrange related emails into "thread-o-logically correct" order. To do this it requires the "In-Reply_to" header to be correctly set, which bugzilla doesn't do. The format of the header is: In-Reply-To: <presvious-message-id>; From email@blah.com on <date> What would be good if it would set the In-Reply-To header to point to the previous comment for that bug so that they will be displayed as a thread.

I'm pretty sure you're confused. What other email message's message-id should bugzilla mail be pointing to? Bugzilla mail does not get generated as a result of an email message; it gets generated as a result of people playing with a webpage.

I'm pretty sure I didn't describe it very well. I mean, when you create a new message (like I'm doing now) this generates an email message. Would it be possible to use the Message-ID header of the email message that gets generated as a result of this? Does that explain what I mean?

No, I'm afraid not. You type in a new comment. This sends out a message. Let's say this message has a message-id of MMM. What do you think the In-Reply-To header of this message ought to say?

I am thinking it should have the Message-ID of the *previous* message that was sent out. That way it will look like a reply to the previous message. So in your example say I type in a comment which results in a message being generated with ID = MMM. Then a little later you reply which generates another message. The In-reply-To header of this message should be MMM under my scheme. Does that explain it?

Well, OK, but it's not likely to happen. In the "new experimental email" world, I send a separate message for each interested person for each bug. That means I would have to keep track of all of those message-ids. I think the benefit here is pretty small, compared to the amount of work and overhead it would take to implement it.

Fair enough I guess. Oh well thanks for listening anyway

tara@tequilarista.org is the new owner of Bugzilla and Bonsai. (For details, see my posting in netscape.public.mozilla.webtools, news://news.mozilla.org/38F5D90D.F40E8C1A%40geocast.com .)

closing as won't fix, sounds to me like newemailtech helps

V.

Verified wontfix? How should the new emailtech help there? I has to be implemented, regardless of old or new emailtech, and I'd really appreciate this feature. And I think it can be done: Bugzilla must generate every Message-Id on its own rather than let the MTA (sendmail, exim, whatever) do it. It must be done in some unique and reproducable way then, like take the bug id, comment date/time and the recipient and build some unique hash from that. On the next comment or bug change the message id for sending out the last (or fist or both?) comment has to be generated again and put in the In-Reply-To and/or References header. Voila.

My thoughts... use the form of (approx example for this message): bugmail-31314-200102211300-<DBID-Recip> Where: bugmail is a static string 31314 is the bug number 200102211300 is a date-time string (YYYYMMDDHHmm) <DBID-Recip> is the DB ID of the recipient. The In-Reply-To header could then be congered up rather then stored (the date-time string could be generated from the bugs.lastdiffed value). This would result in no extra storage overhead and only a little processing.

this would allow the newemailtech to change the subject of the bugmail but still allow threading e-mail clients (including mozilla) to thread together bugmail from the same bug...

Alec: excatly that's what is bugging me, too. I'm sorting my bug-mailbox by subject to get it as close as possible to threads, but when the summary changes, the sorting gets messed up, and the first email for a bug ("New") comes after all the other ones ("Changed"). Jake: that's exactly what I meant, only that you made it clearer than I did. :) I'd like to implement that, I only don't know when I have the time for it. But if anyone else wants to do it, feel free, but please assign this bug to you then (or tell me by some other means).

*** Bug 52128 has been marked as a duplicate of this bug. ***

I guess you're supposed to create random MsgIDs, but maybe we could make the first part random, and make the second part be determined by the bug number - that way we wouldn't need to track the MsgIDs. It's not like there's going to be many other msgs coming from Bugzilla.

-> Bugzilla product, Email component, reassigning.

*** Bug 78117 has been marked as a duplicate of this bug. ***

> I guess you're supposed to create random MsgIDs NO, you are supposed to generate *unique* MsgIDs. That's usually done with randomness, but it's not at all required. Just make sure to have "@bugzilla.mozilla.org" (for b.m.o) at the end and that you don't generate the same "local" part for 2 different comments. I'm not sure, if 1 comment can cound as one unique email or if each email for the several recipients counts as separate msg for RFC822. (Since bugzilla generates one email for each recipients, and they differ.) In any case, I think that this bug is a bad idea, because if you point In-Reply-To to the previous comment, most mailers (incl. Mozilla) will indent each comment further. Now, imagine a bug with 50 comments - the last one will be indented 50 times, unless the mailer has some specific provision to prevent that.

What we've done (at least temporarily) is add: In-Reply-To: <bug-%bugid%@bugzilla.ximian.com> To each bug email. This is more or less blatant violation of RFC 822, as it is in no way unique per email. Furthermore, it does not indent in a chain.* However, it 'works for us' because mailers that follow the jwz threading algorithm**,*** will show all of the emails with this identical tag as children of the first one. While imperfect, this allows sane clients that have collapse and select threads commands to do useful things with the thread as a whole and eliminates the need to sort by subject in a large bugzilla folder, which (IMHO) are the biggest reasons to have threading for bugzilla emails. I'm not sure how to go about implementing this at b.m.o.; there is no code to be modified, just editparams.cgi. *Which, Ben, I think is desirable- the whole /point/ of threading is to get indentation and make the order of things useful and sane.**http://www.jwz.org/doc/threading.html ***evolution does; no clue if mozmail does or not

Another point: Some (most?) mailers only allow to expand a whole thread via a keyboard shortcut or (sub)menu (or not at all). It would be very time-consuming to get to the later msgs. > the whole /point/ of threading is to get indentation Yes, but Mozilla comments are no threads, they are linear. I don't think that every-increasing indention is very useful or makes anything clearer. > make the order of things useful and sane If you sort by time or order recieved (the second criteria in all mailers I know, if the first sort criteria is Subject), msgs are sorted correctly.

<shrug> doesn't really matter to me; since what I need can be done without patching anything it's not really a big deal if this goes in upstream. So it's not really worth arguing about much. FWIW, it probably wouldn't be too hard for someone to patch this in as a preference.

Changing default owner of Email Notifications component to JayPee, a.k.a. J. Paul Reed (preed@sigkill.com). Jake will be offline for a few months.

I will attach a patch against 2.16 for processmail that does the In-Reply entry into the mail in a very simple way. On new mails, the message ID for the mail is created from the creation timestamp, the number and the reporter email of the bug. The fields are concatenated by a dot. On bugs that are not new, not the message ID but the 'In-Reply-To:' field is set with that string. That makes that all messages that follow up on a new bug are threaded under the NEW mail on the same first level. In Pine that looks like: +1 Sep 17 bzdaemon@suse.de (1) [Bug 18199] New: Test Indentation +2 Sep 17 bzdaemon@suse.de (1) |-[Bug 18199] Test Indentation +3 Sep 17 bzdaemon@suse.de (1) \-[Bug 18199] Test Indentation The following two lines need to be added to the newchangedmail-parameter to make the patch working: Message-ID: %msgid% In-Reply-To: %inreplyto%

Patch for Bugzilla 2.16

I have created a new patch that patches 2.17.2, and it seems to work (msg id and x reply to have a nice autogenerated msg). There is one problem though - postfix seems to be overriding the msgid that processmail sets. Any ideas?

Comment on attachment 99515 [details] [diff] [review] Patch for processmail adding In-Reply-To processmail will not exist after bug 124174 goes in, but if you want to attach a ported patch to the new Bugzilla::BugMail module (check 124174), that'd be great.

Comment on attachment 113606 [details] [diff] [review] processmail patch for 2.17.2 processmail will not exist after bug 124174 goes in, but if you want to attach a ported patch to the new Bugzilla::BugMail module (check 124174), that'd be great.

*** Bug 201786 has been marked as a duplicate of this bug. ***

*** Bug 205392 has been marked as a duplicate of this bug. ***

This goes the same way the previous patched do, giving a Message-ID for the new-bug-message and an In-Reply-To for any additional comments, but instead of a timestamp, the bug id is used. The message id is built up like this: <bug-%bug_id%-%user_id%@%sitespec%> where %sitespec% is Param('urlbase') minus the protocol and a trailing slash.

Comment on attachment 162075 [details] [diff] [review] Patch against head sorry, but message-id's need to be globally unique. rfc 2822 section 3.6.4 "The message identifier (msg-id) itself MUST be a globally unique identifier for a message. The generator of the message identifier MUST guarantee that the msg-id is unique." and "A message identifier pertains to exactly one instantiation of a particular message; subsequent revisions to the message each receive new message identifiers."

(In reply to comment #31) > sorry, but message-id's need to be globally unique. Sure, but the patch does that, not?

Given that bugzilla has no way (that I know of) of generating such a unique ID and storing it for responses, and given that this approach, while not standards compliant, is useful enough that gnome, ximian, and redhat (three of the largest bugzilla installations I know of) use my original patch or a variant thereof, I don't think that standards-compliance in this case should be grounds for a patch rejection.

Comment on attachment 162075 [details] [diff] [review] Patch against head ah, my bad, sorry. i missed that message-id is only used for new messages, so it _will_ be unique. r=glob

Comment on attachment 162075 [details] [diff] [review] Patch against head The regexp to produce sitespec will break any sites that use port numbers... http://mysite.mydomain:8000/bugzilla Also, what happens when people are added to a bug after it is new?? You can fix the sitespec problem by using bug-$id-crypt(urlbase)@domain On the second issue, there is a dilemma.... you could use a Message-ID when someone is initially added to a bug, but you would have to check to make sure that they were not previously sent mail on that bug. (difficult) Also, what about flag-mail?

> what happens when people are added to a bug after it is new? Then they are missing the original mail, but get the "responses" to it. Mail/News readers have to deal with situations like this anyways, e.g. for usenet or when new people are added to the cc.

(In reply to comment #36) Fair enough. If you fix the sitespec parsing so it wont break with port numbers, this will be a winner. I would rather this address the review requests as well, but that is not a showstopper.

(In reply to comment #35) > The regexp to produce sitespec will break any sites that use port numbers... > http://mysite.mydomain:8000/bugzilla > You can fix the sitespec problem by using > bug-$id-crypt(urlbase)@domain I could fix the port number issue by adding a '?' in the regexp, but I like your crypt()-idea better. I'll do this tomorrow. > Also, what about flag-mail? Flag-mail is not addressed by this patch. Let's do this in a separate bug. (In reply to comment #33) > this approach, while not standards compliant I can't see that. A Message-ID is recommended, but not required. We're issuing globally unique IDs which conform to rfc2822's dot-atom. What am I missing? I'd like to do it in a standard compliant way...

why not cgi-enode urlbase use that as the sitespec?

Using crypt(Param('urlbase')) to make the Message-ID unique.

This patch uses url_quote().

This patch generates an easily readable Message-ID.

Someone told me about this bug after I had already written this into the SpamAssassin Bugzilla earlier today. This is pretty straighforward: - generate a single message-id for the new bug - changes to the bug get unique message-ids and are replies to the original new bug message I only add threading to change messages (since those are the only ones we have going to our list, but it would not be hard to do the same for others.

Comment on attachment 162176 [details] [diff] [review] Patch 1.2-pretty I don't like the idea of cutting out the port number. That adds a risk of the Message ID not being unique if a site runs multiple Bugzillas on different ports (I've seen it happen). The port number should be in cluded in whatever algorithm is used to generate the sitespec part of the message ID.

Comment on attachment 167229 [details] [diff] [review] different patch I'm going to leave full review of this to the folks who've already been doing reviews on this bug. I like the way you're doing the header replacement in the message template in this patch, but I'm leaning towards the other patch for my favorite way of generating the message ID. Perhaps we can combine the two.

> I don't like the idea of cutting out the port number the port isn't being removed, it's being prepending to sitespec if it's provided.

Comment on attachment 162176 [details] [diff] [review] Patch 1.2-pretty For urlbase http://foo.bar:8080/bugzilla/, bugid 31314, receiver user id 4711, the Message-ID will be <bug-31314-4711-8080@http.foo.bar/bugzilla/>. For urlbase http://foo.bar/, bugid 31314, receiver user id 4711, the Message-ID will be <bug-31314-4711@http.foo.bar/>. For urlbase https://foo.bar:4433/bugzilla/, bugid 31314, receiver user id 4711, the Message-ID will be <bug-31314-4711-4433@https.foo.bar/bugzilla/>. Requesting review again. Personally, I don't like the concept of cutting header lines back out... This is sooo much cleaner with templates :-)

Comment on attachment 162176 [details] [diff] [review] Patch 1.2-pretty removing my review-, as the reason for it was misplaced

So... I understand we need to decide on the way we want to do this. Opinions, everybody? To remove any doubts about my personal opinion, I like attachment 162176 [details] [diff] [review] best the way it is ;-)

Comment on attachment 167229 [details] [diff] [review] different patch This probably isn't the best solution anyway, IMHO, but if this method is chosen, this patch isn't gonna work for it. >+ # remove In-Reply-To: and References: headers if they're empty >+ $msg =~ s/^(?:In-Reply-To|References):\s*$//m; I really don't see where Refreneces is coming from... >+Message-ID: %messageid >+In-Reply-To: %inreplyto These variables need a % after them, too :) Probably not worth doing another patch for this unless this ends up being the chosen method.

Comment on attachment 162176 [details] [diff] [review] Patch 1.2-pretty I think this one is my favorite. >+my $sitespec = '@'.Param('urlbase'); >+$sitespec =~ s/:\/\//\./; # Make the protocol look like part of the domain >+$sitespec =~ s/^([^:\/]+):(\d+)/$1/; # Cut out a port number >+if ($2) { >+ $sitespec = "-$2$sitespec"; >+} >+ You may want to consider making the comment a little clearer that you're not cutting out the port number, you're really just relocating it. >--- orig/defparams.pl 2004-09-26 02:01:30.000000000 +0200 >+++ patched/defparams.pl 2004-10-14 15:32:58.000000000 +0200 >@@ -671,6 +671,7 @@ > default => 'From: bugzilla-daemon > To: %to% > Subject: [Bug %bugid%] %neworchanged%%summary% >+%threadingmarker% > X-Bugzilla-Reason: %reasonsheader% > > %urlbase%show_bug.cgi?id=%bugid% You also might wanna add %threadingmarker% to the descriptive text for the Param.

Thanks for the review, Jake. Addressing comments.

Comment on attachment 170331 [details] [diff] [review] Patch 1.3 I think a message id of: bug-6-1@http.www.steenhagen.us/bugzilla/tip Looks a little funny, but it works. (If you're curious, I think something like bug-6-1-bugzilla-tip@http.steenhagen.us would look better, but that's OK.)

(In reply to comment #53) > (From update of attachment 170331 [details] [diff] [review] [edit]) > I think a message id of: > bug-6-1@http.www.steenhagen.us/bugzilla/tip > > Looks a little funny, but it works. (If you're curious, I think something like > bug-6-1-bugzilla-tip@http.steenhagen.us would look better, but that's OK.) I agree this would look better, but I think we need to keep the slashes to be able to distinguish between bugzilla/tip and bugzilla-tip. That would make it bug-6-1-bugzilla/tip@http.steenhagen.us. This is imo unfortunately as ugly as bug-6-1@http.www.steenhagen.us/bugzilla/tip :(

(In reply to comment #54) > able to distinguish between bugzilla/tip and bugzilla-tip. I think the chances of somebody intentially having two installs with URLs that similar are pretty slim, but I digress :).

> I think we need to keep the slashes to be > able to distinguish between bugzilla/tip and bugzilla-tip. The likelyness is far larger that some mail client won't like slashes in msgids.

I agree the chances of a clash are rather far on the slim side :) Let's have it as pretty as possible. Saying s/\//-/g is simple enough, but we're talking about a field almost never seen by any user, so when it comes to it, personally I value safe over pretty here :) RFC2822 says msg-id = [CFWS] "<" id-left "@" id-right ">" [CFWS] id-left = dot-atom-text / no-fold-quote / obs-id-left id-right = dot-atom-text / no-fold-literal / obs-id-right dot-atom-text = 1*atext *("." 1*atext) atext = ALPHA / DIGIT / ; Any character except controls, "!" / "#" / ; SP, and specials. "$" / "%" / ; Used for atoms "&" / "'" / "*" / "+" / "-" / "/" / "=" / "?" / "^" / "_" / "`" / "{" / "|" / "}" / "~" no-fold-quote = DQUOTE *(qtext / quoted-pair) DQUOTE no-fold-literal = "[" *(dtext / quoted-pair) "]" so I expect mail clients not to bork on slashes.

Targeting bug to 2.20 since the 2.20 feature freeze was canceled.

Marc, is this ready for checkin? Your comment #57 is slightly confusing to me; not sure if you're saying it needs something else, or you want to keep existing features. Let me know if it's ready, and I'll get it checked in.

(In reply to comment #59) Shane, this can go in as is. I meant to clarify that this is the case :)

Checking in defparams.pl; /cvsroot/mozilla/webtools/bugzilla/defparams.pl,v <-- defparams.pl new revision: 1.143; previous revision: 1.142 done Checking in Bugzilla/BugMail.pm; /cvsroot/mozilla/webtools/bugzilla/Bugzilla/BugMail.pm,v <-- BugMail.pm new revision: 1.22; previous revision: 1.21 done

Thank you all. Now don't forget to put %threadingmarker% into your newchangedmail parameters on your HEAD installs :)

So, from the last comment, I take it that there's some things that people have to do take advantage of this enhancement? Sounds like new information that could go in the documentation... I'm marking 'Documentation?' for this bug to indicate that it needs some explanation, but it would be of *immense* help to the documentors if you could write an explanation of this enhancement and what needs to be done to make use of it. It doesn't have to be in SGML, and you don't have to say where it's going to go in the docs... just write it in English, store it in a .txt file, and attach it to this bug. That way when someone comes to do the docs, they'll already have a head start. Thank you!

This is perhaps a bit late, but shouldn't threadingmarker be on by default? seems to me there is no downside...

Explanation on what needs to be done when upgrading.

(In reply to comment #64) > This is perhaps a bit late, but shouldn't threadingmarker be on by default? > seems to me there is no downside... It is active for new installs. For existing installs, it would require Config.pm to mangle the newchangedmail parameter. This is theoretically possible, but I don't recommend going this way because it's free-text, and I wouldn't like to take the slim chance to break some site's parameter. So imo, we should cover this by an update note. I'm ready to get overridden now :)

Moreso than documentation, it should be relnoted.

*** Bug 240902 has been marked as a duplicate of this bug. ***

Added to the release notes in bug 286274.

As this parameter is dead and has been replaced by a template, no action is required anymore to make it work. So no need to document this.