Closed Bug 110692 Opened 23 years ago Closed 19 years ago

Properly encode subjects in newchanged mails.

Categories

(Bugzilla :: Email Notifications, defect, P1)

Product:
Bugzilla
Component:
Email Notifications
Version:
2.15
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Bugzilla 2.22

People

(Reporter: marcus, Assigned: glob)

References

Details

(Whiteboard: i18n [blocker will fix])

Attachments

(1 file)

Patch against 2.17 Head for iso subjects
(deleted), patch
preed
: review-
Details | Diff | Splinter Review 
I'm using both Bugzilla and Courier on an in-house-system. After entering a Bug,
I get the following message:

Bug 2 posted
550-The headers in this message contain 8-bit text, which violates RFC 2047.
550-Sorry, I cannot accept any mail without the required MIME headers. 550 See
for more information.
*** Bug 110691 has been marked as a duplicate of this bug. ***
What mail program are you using, and what version?  Any special configuration
options?
I'm using courier 0.35

Btw, the mail message are configurable by the administrator of Bugzilla? And he
can put any text he wants into them, no matter if the characters used are within
the 7-bit-ASCII range. If I read RFC 2045 (Message Header Extensions for
Non-ASCII Text) correctly, any characters which appear in the message header and
are not part of US-ASCII, have to be encoded by the MUA (which in this case
would be Bugzilla). Currently, we just read the mail-messages from localconfig,
replace some keywords, and pass the result to the MTA as is.
I don't see anything obviously in high-ASCII in the default, so I'm thinking
it's the "to" or "subject" headers (probably subject) which take data from
Bugzilla.

How would we appropriately encode these?

What summaries are you entering in?  Are you entering high-ASCII characters into
them?  Does it go away if you don't?
Oh, you simple have to use german umlauts like הצ� in your summary, to get
high-ASCII characters. So this is an internationalization problem.

For the appropriate encoding, please have look at RFC 2045. For example, the
encoding for 'A' in quoted-printable would be =41
Priority: -- → P2
Summary: Default %newchangedmail% doesn't comply with RFC 2047 → Properly encode subjects in newchanged mails.
Target Milestone: --- → Bugzilla 2.18
Hi KaiRo,

As you requested at FOSDEM, I added you to this bug.

Marcus
Have also a look at bug 126266.

For emails this creates this header:
 MIME-Version: 1.0
 Content-Type: text/plain; format=flowed; charset=%encoding%
 Content-Transfer-Encoding: 8bit
while this isn't necessarily as good as MIME encoding, this produces at least
correct output.
(As a bonus you also get the charset for HTML.)

Changing default owner of Email Notifications component to JayPee, a.k.a.
J. Paul Reed (preed@sigkill.com).  Jake will be offline for a few months.
Assignee: jake → preed
This becomes a problem for us when bug summaries have Japanese text in them.
Actually anything out of 7-bit ASCII range, since all of our data is stored in
UTF-8.

Has anyone looked at fixing this yet? If not I'm going to in the next couple of
days. The lack of a patch indicates 'no, nothing has been done.', but I wanted
to ask.
Have at it.

You might want to hold off on this until all emails get templatised as the new
development cycle ramps up.  This might automatically solve the problem.  The
easy and reliable solution would be to escape the entire email (for all emails)
after it's constructed, but I'm not sure if there are any problems precluding
that approach.
There are two sides to this: the message body itself can be encoded in base64 
or quoted-printable. The subject line uses the MIME encoding: a bit different.

Headers and bodies will probably be separate templates, assuming the headers are
going to be templates at all, so that's probably not an issue.
*** Bug 147810 has been marked as a duplicate of this bug. ***
*** Bug 179046 has been marked as a duplicate of this bug. ***
I've been running into this with bugzilla.mozilla.org and Postfix's
strict_7bit_headers option. With it enabled, I no longer get bugzilla emails.
Instead, I see this:

reject: mime-error improper use of 8-bit data in message header: Received: (from
nobody@localhost)??by mothra.mozilla.org with ? id h09C0ca09098;??Thu, 9 Jan
2003 04 from mothra.mozilla.org[207.200.81.216]; from=<nobody@mozilla.org>
to=<bugzilla@slamb.org>

From looking at old ones, it's easy to see why. Every email includes a Received:
header like this:

Received: (from nobody@localhost)
	by mothra.mozilla.org with œ id h072KKM28806;
	Mon, 6 Jan 2003 18:20:20 -0800 (PST)

What's the "œ" doing there?
I've been strugglig for some time now to get norwegian characters to display 
correctly in the mail Subject of mails from bugzilla (using 2.16.2 here).

Here is my working solution (to processmail):

Add:

use Encode qw/encode decode/;

Change line reading:

$substs{"summary"} = $values{'short_desc'};

to

$substs{"summary"} = encode('MIME-Header', $values{'short_desc'});

As this is my first go at this - please forgive a possible 'plain stupidness'. 
Anyways - it works for me ;)

unfortunatelly i was not able to make work encode for perl 5.6.1 (using
encode::compat)

could somebody help me, please?
Yeah, I just noticed this self-same issue with Bugzilla 2,14 (must pester the 
server manager to upgrade soon).

It would seem the problem is that the Subject: lines of emails aren't being 
checked for high-value (>127) characters and no MIME-escaping is occurring as a 
result.  This'd be an i18n bug, for sure.  RFC 2047 details the howto at length 
but, basically, you want the Subject: line to look something like =?iso-8859-1?
q?this=20is=20some=20text?= (where the "=20"s are spaces, natch, though space 
can also be translated to an underscore).
Here is a patch that allows international characters in subjects which seems at
least to fix partially the problems reported in this bug.. I've created a new
variable %isosummary% because the %summary% variable is used elsewhere.. so
existing installs needs to change %summary% to %isosummary% in their existing
newchangedemail configuration.

It uses MIME::Words
Comment on attachment 132868 [details] [diff] [review]
Patch against 2.17 Head for iso subjects

Asking for review
Attachment #132868 - Flags: review?
works for me! it solved all my problems with summarry in latvian.
My issues with this patch:

-- It requires another module; that may be inevitable, but...
-- This probably isn't going to make it into 2.17.5... mainly because there's
really no reason for it to go in, and we're trying to make 2.17.5 stable; given
that:
-- This patch is against the old defparams.pl way of doing things; TT may
actually take care of this for us (it may not; I haven't checked), but with bug
84876 (see today's comment of a few minutes ago) and bug 100089 we're using TT
JP, can you check out whether TT takes care of this for us, and if so, can you
set a dependency to your bugs (that is, if you're actually *going* to finish
then :-)
Comment on attachment 132868 [details] [diff] [review]
Patch against 2.17 Head for iso subjects

Per Jaypee's comments, pushing the review request to his queue.
Attachment #132868 - Flags: review? → review?(preed)
Comment on attachment 132868 [details] [diff] [review]
Patch against 2.17 Head for iso subjects

If we're going to do this, you might as well do it the right way and make it
part of the email template stuff in bug 84876. Yes, it's not in yet, but the
subject stuff that's there isn't going to change, and it should be going in
shortly now (where shortly is actually something measurable).

I also don't like the idea of requiring MIME::Tools, but...
Attachment #132868 - Flags: review?(preed) → review-
Wow, I completely missed this bug happening...

JayPee: we have no choice but to require MIME::Tools if we want
standards-compliant emails and still do i18n (unless you feel like reinventing
the wheel).  If you read existing comments in bug 126266, you'll see this has
been the plan for quite some time.  That new module requirement is inevitable.
Depends on: 84876, bz-charset
Comment on attachment 132868 [details] [diff] [review]
Patch against 2.17 Head for iso subjects

The proper way to do this, I think, is to modify the existing %summary%
replacement instead of making a new replacement.  That way existing sites get
it without having to modify their mail parameter.  (defparams.pl is only the
defaults)
I would agree, except that the %summary% replacement is if I recall correctly
used in cases where the 'iso' encoding should not happen (inside the email for
exemple)..
As I understand it, you need to encode the header lines all in the same way, and
that way is the same for all headers.

As such to me the obvious implementation is to provide a filter over the
template output and leave everything unencoded in the template.  Or
alternatively get a module that automatically encodes headers.

Expecting this to be done in templates is just wrong, given it is compulsory,
makes the templates more complicated and people are just going to forget it or
accidentally remove it.

As long as we don't do something silly like put both the headers and body in the
same template this shouldn't be a problem.
Yeah, Matty's right.  The new email system that JayPee is doing should be able
to handle it that way pretty easily, since the headers are going to be done
separate in the "new world"
Severity: normal → critical
OS: Windows 98 → All
Priority: P2 → P1
Hardware: PC → All
Whiteboard: i18n
Target Milestone: Bugzilla 2.18 → Bugzilla 2.20
Blocks: 84876
No longer depends on: 84876
OK, my developers at Kerio (in the Czech Republic) want this, so I'm taking it.
Assignee: preed → mkanat
Severity: critical → major
my latest patch on bug Bug 126266 resolves this issue.
we can probably make this bug as a dupe.
Assignee: mkanat → bugzilla
Whiteboard: i18n → i18n [blocker will fix]
Now that the blocker is going to 2.22, I'd imagine this is going to 2.22 also.
Target Milestone: Bugzilla 2.20 → Bugzilla 2.22
Fixed by blocking bug 126266.
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → FIXED
QA Contact: matty_is_a_geek → default-qa
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: