Closed Bug 1731007 Opened 3 years ago Closed 3 years ago

SMTP 1000-character line limit violated by quote - causes invalid DKIM signatures on outbound mail.

Categories

(MailNews Core :: Networking: SMTP, defect)

Thunderbird 91
defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: mozbz, Unassigned)

References

Details

Attachments

(5 files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

When quoting a received email in plaintext where the original email had a line that hit the 1000-characther SMTP line-length limit (including the CRLF), Thunderbird will happily send a line that is longer than 1000 characters over the wire to the configured SMTP server, violating the RFC.

While this probably mostly shouldn't be an issue, in the case of Postfix + a DKIM-signing milter, this may cause the generated DKIM signature to be invalid as the MTA may truncate the line after the DKIM signature is generated [1].

In my specific case of a pretty standard Thunderbird + Postfix + opendkim setup, this does generate invalid DKIM signatures on outbound mail.

[1] https://marc.info/?l=postfix-users&m=163176643901328&w=2

Thanks for reporting, can you provide an eml file so that it's easier to try to reproduce this bug?

Blocks: tb91found
Component: Untriaged → Networking: SMTP
Product: Thunderbird → MailNews Core
Attached file Test Email.eml (deleted) —
Attached file Reply To Test Email.eml (deleted) —

Attached two files - one is a test email received by Thunderbird, the second is the reply to said test email (by simply hitting the reply button in plaintext mode). The reply email is sent back to the server with an invalid line length (and you'll note it was sent to a DKIM validator, which confirmed the DKIM signature that it ended up with was invalid).

Attachment #9241635 - Attachment mime type: message/rfc822 → text/plain
Attachment #9241634 - Attachment mime type: message/rfc822 → text/plain

Thanks, can you please download a nightly from https://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-central-l10n/thunderbird-94.0a1.en-US.linux-x86_64.tar.bz2 and try to reproduce it? I think it's fixed in bug 1729437

Flags: needinfo?(mozbz)

Confirming based on reporter's expertise. This is actually pretty serious when it happens. Ping thinks it might be fixed by bug 1729437. Let's hear from Matt (reporter) if that's true...

Severity: -- → S2
Status: UNCONFIRMED → NEW
Ever confirmed: true

(In reply to Thomas D. (:thomas8) from comment #6)

Confirming based on reporter's expertise. This is actually pretty serious when it happens. Ping thinks it might be fixed by bug 1729437. Let's hear from Matt (reporter) if that's true...

So, was it? Should this bug be made a duplicate?

Flags: needinfo?(bugzilla2007)

Hi, sorry for the delay on this one.

Anyway, the immediate bug does appear to be fixed in 91.2.0, but there is a second related violation of the 1000 character limit which is still broken - I cannot attach a .eml which violates the 1000 character limit to a mail.

Flags: needinfo?(mozbz)

You're right, it's an old problem. Since bug 1255071, eml attachments are injected into the final message without any transformation. Seems gmail web does the same. I guess it's like this because not many people will attach "invalid" eml files. Even if they do, sending/receiving works in most cases.

Yes, today I discovered: (a) vger.kernel.org doesn't limit line lengths, then when trying to forward the .eml to the postmaster there, (b) Thunderbird doesn't handle that .eml correctly, and (c) iOS Mail also doesn't handle that .eml correctly. Seems this is kinda broken everywhere, but, hey, doesn't mean it isn't worth fixing :)

According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4

Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".

So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.

RFC 1341 seems to say this, however, implying the "correct" answer would be "binary".

        The difference between  "8bit"  (or  any  other  conceivable
        bit-width  token)  and  the  "binary" token is that "binary"
        does not require adherence to any limits on line  length  or
        to  the  SMTP  CRLF semantics, while the bit-width tokens do
        require such adherence.  If the body contains  data  in  any
        bit-width   other  than  7-bit,  the  appropriate  bit-width
        Content-Transfer-Encoding token must be used  (e.g.,  "8bit"
        for unencoded 8 bit wide data).  If the body contains binary
        data, the "binary" Content-Transfer-Encoding token  must  be
        used.

(not that "binary" will solve the issue here, but maybe it will encourage BDAT usage, dunno if Thunderbird supports BDAT instead of DATA)

(In reply to Ping Chen (:rnons) from comment #11)

According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4

Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".

So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.

I think you have misunderstood. The paragraph refers only to the parent structure of a multipart MIME message, because this could itself contain headers again.

The body of a single MIME part, which does not contain any other MIME structure, may very well be BASE64 or similar encoded.

IMHO this encoding may also be changed. If I remember correctly, this results from the SMTP RFC. There it is mentioned that transport servers are allowed to change the encoding if it is necessary e.g. for the 7-bit transmission.

I'm not sure if you're subscribed to mailop, but Brandon Long from Google seems to have agreed with Ping's understanding at https://list.mailop.org/private/mailop/2021-October/020231.html saying this about GMail:

Also, there is a tension in the rfc's here, because you aren't allowed to
encode message/rfc822, so you would have to edit the attached file.

We used to base64 encode the message/rfc822 anyways, but we had enough
receivers not able to handle that, so we had to revert. We debated just
using message/global instead, but I think even fewer receivers understood
that.

Of course, we also will happily use BDAT if the remote server supports it,
which doesn't have the line length limitation.

Attached file Test Email_BASE64.eml (deleted) —

(In reply to Matt from comment #15)

I'm not sure if you're subscribed to mailop, but Brandon Long from Google seems to have agreed with Ping's understanding at https://list.mailop.org/private/mailop/2021-October/020231.html saying this about GMail:

No, so I do not know the context.

But again: It is not allowed to pack the complete message/rfc822 message in BASE64 encoding.

But it is allowed to change the body of the message.
I changed attachment 9241634 [details] as an example.

No, so I do not know the context.

There's not a lot of context you're missing, basically just this current discussion. I read his response as "we used to violate the RFC and encode the full attachment as base64, because doing otherwise is somewhat nonsensical, but receivers had problems with it, so GMail stopped doing that", which I figured was useful context for this discussion :).

Attached file RFC2045_violation.eml (deleted) —

This is not allowed!

Attached file RFC2045_OK.eml (deleted) —

Multipart message with changed body type - this is OK

Attachment #9245765 - Attachment mime type: message/rfc822 → text/plain
Attachment #9245766 - Attachment mime type: message/rfc822 → text/plain

(In reply to Alfred Peters from comment #14)

(In reply to Ping Chen (:rnons) from comment #11)

According to https://datatracker.ietf.org/doc/html/rfc2045#section-6.4

Certain Content-Transfer-Encoding values may only be used on certain
media types. In particular, it is EXPRESSLY FORBIDDEN to use any
encodings other than "7bit", "8bit", or "binary" with any composite
media type, i.e. one that recursively includes other Content-Type
fields. Currently the only composite media types are "multipart" and
"message".

So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.

I think you have misunderstood. The paragraph refers only to the parent structure of a multipart MIME message, because this could itself contain headers again.

The body of a single MIME part, which does not contain any other MIME structure, may very well be BASE64 or similar encoded.

IMHO this encoding may also be changed. If I remember correctly, this results from the SMTP RFC. There it is mentioned that transport servers are allowed to change the encoding if it is necessary e.g. for the 7-bit transmission.

I think there is no misunderstanding. The paragraph says for any MIME part, when Content-Type is multipart/* or message/*, the Content-Transfer-Encoding can only be 7bit, 8bit or binary.

I see you are suggesting validating and normalizing sub parts of message/* when user attaching eml, it is a bit too much for me. That requires a lot of efforts and still there will be eml files totally broken and can't be fixed. There can also be cases that we should not change headers of sub parts, because the original can't be restored on the receiving side.

(In reply to Ping Chen (:rnons) from comment #20)

(In reply to Alfred Peters from comment #14)

(In reply to Ping Chen (:rnons) from comment #11)

So we can't encode it with base64. Inserting line breaks may work in some cases, but I'm not sure.

I think you have misunderstood. [...]

I think there is no misunderstanding. The paragraph says for any MIME part, when Content-Type is multipart/* or message/*, the Content-Transfer-Encoding can only be 7bit, 8bit or binary.

Well, then I misunderstood you - sorry about that.

I see you are suggesting validating and normalizing sub parts of message/* when user attaching eml, it is a bit too much for me. That requires a lot of efforts and still there will be eml files totally broken and can't be fixed. There can also be cases that we should not change headers of sub parts, because the original can't be restored on the receiving side.

Yes, you are right, if the attachment itself is a multipart message, it will be difficult. However, it would be the cleanest solution.

I don't see the binary content type as a solution. Is this supported by any email reader at all? I have not encountered such a mail yet. But that doesn't have to mean anything either.

As a stopgap, we could display a warning, and leave it up to the user to decide whether to attempt to send, or to customize the email. Whereby probably the fewest are able to do so.

Sound like this is wontfix: no real solution in sight, for pretty low benefit even if a solution would be found.

Summary: SMTP 1000-character line limit violated by quote → SMTP 1000-character line limit violated by quote - causes invalid DKIM signatures on outbound mail.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(bugzilla2007)
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: