We have some pretty arcane options of users that "cannot receive HTML email". This is simply not a a thing in 2021. You could not use such a program to read your every day emails.

This is Preferences | Send options | When sending messages in HTML format .... + related dialog to ask which format to send in - askSendFormat.xhtml

Instead of putting effort into converting/maintaining this we should remove it.

Besides it's obvious lack of real world connection at this day, it is also a very confusing user experience since people cannot realistically understand that messages are created once and sent to all, meaning if you had a mix of recipients, you potentially downgrade for all.

https://searchfox.org/comm-central/rev/ddd293b13d7052f64e51dca70678bfdf88fe2088/mail/components/compose/content/MsgComposeCommands.js#4819

From past experience with this, I support the removal of this feature wholeheartedly.

• If we remove this, do we need to give users some sort of plain vanilla choice for their default HTML compose format between HTML-only vs. Text+HTML?
• Is Text+HTML still a format which we need to offer?

(In reply to Thomas D. (:thomas8) from comment #2)

• If we remove this, do we need to give users some sort of plain vanilla choice for their default HTML compose format between HTML-only vs. Text+HTML?

I don't see the need for it. We convert to plain text when possible already, and the pref for what to do when we can't do that would still exist only now just a hidden pref.

• Is Text+HTML still a format which we need to offer?

Good question... but will be hard to determine. In the end, sending the text part as well is not a huge overhead so we probably want to continue doing so.

@mkmelin

Is your idea to remove the "Send Options..." dialog entirely? But still keep the mailnews.sendformat.auto_downgrade, mail.default_html_action mailnews.html_domains and mailnews.plaintext_domains as preferences and their corresponding functionality?

Should we keep a UI element for mailnews.sendformat.auto_downgrade?

And, if we get rid of the askSendFormat.xhtml dialog, then the AskUser preference for mailnews.default_html_action (https://searchfox.org/comm-central/rev/ce083e3d3bd470e28981aceeb1028807ffd15a78/mailnews/compose/public/nsIMsgCompose.idl#49) will have to be removed. And for cases where mailnews.default_html_action has already been unset or set to AskUser by the user, we will need to change it to a different value. Specifically here https://searchfox.org/comm-central/rev/ce083e3d3bd470e28981aceeb1028807ffd15a78/mailnews/compose/src/nsMsgCompose.cpp#4724 Would you want it to fallback to Both (i.e. Text+HTML)?

Is it ok to simply remove an entry from the nsIMsgCompSendFormat idl and ignore a set AskUser mailnews.default_html_action preference? Or is there some specific conversion I would need to do?

Thanks for the ping.

mailnews.sendformat.auto_downgrade should stay as it is, but there would be no UI for it anymore.

mail.default_html_action, for the value of "ask", instead use "both"
For migration: please remove the value from the .idl, and add a migration where you move the pref set to 0 to 2 instead.
https://searchfox.org/comm-central/source/mail/modules/MailMigrator.jsm#526

mailnews.html_domains and mailnews.plaintext_domains: I think they should be completely removed, since realistically not being able to process html email on a domain would not work out for anyone.

The per-use preference in the address book: I think we can punt on removing this for now, so that the preference (mailnews.sendformat.auto_downgrade, now hidden) for what to do if a format is specified still applies . But eventually I'd like to remove that as well.

How does this affect sending mail to users of "plain text" clients like Mutt (https://en.wikipedia.org/wiki/Mutt_(email_client))? Some of those complain quite vocally when receiving HTML-only mail. How does a TB user communicating with such people ensure that they always get plain text (or plain text + HTML)? Setting the desired delivery format every time when sending a message can easily be forgotten. Chances are that because TB changes its behavior, they won't be convinced to stop using their "arcane" client.

Unless the user specifies otherwise, our default is to send both plain text and html (if not pure plain text is sent). Mutt can be set to either use the plain text part, OR (apparently), to convert the html part and show that. https://wiki.archlinux.org/title/Mutt#Viewing_HTML

Well, I've chosen to send HTML-only (non-default) since IMHO it makes no sense to send both for most recipients. So I'd have to switch to plain+HTML for that one recipient only which can easily be forgotten. Please don't remove a feature that is in use, even if only used by a minority. I don't care about the UI in comment #1, but you should provide a facility that certain recipients can additionally receive plain text if the default is HTML-only.

@Magnus: Please make sure to CC me, and ask for my opinion, on changes for code that I wrote. Politeness and team cooperation mandates that.

Depends on D136007

The nsIMsgCompSendFormat.AskUser enum value had a dual role in the code before:

1. The return value from nsMsgCompose::DetermineHTMLAction when choosing just HTML vs plain text could not be determined and "mail.default_html_action" was set to 4 ("AskUser").
2. Used in connection to the compose window #format_auto and MsgCompFields.deliveryFormat field to indicate that the format should be automatically determined (i.e. to use nsMsgCompose::DetermineHTMLAction), rather than an explicit format selected by the user in #outputFormatMenu.

Only case 2 is still in use, so nsIMsgCompSendFormat "AskUser" was renamed to "Auto".

"mail.default_html_action" now only accepts 1 (PlainText), 2 (HTML) and 3 (Both). And previous AskUser values are treated as Both instead.

Depends on D136008

Hey Henry, we don't know each other. Nice to meet you. Welcome to Thunderbird! I've written this code about 20 years ago. (It was one of my first contributions to Thunderbird.)

• I think it's fine to remove the domain-based options, which need a lot of UI surface and also backend logic.
• I would keep the general "Plaintext vs. HTML-only vs. both" preference in the UI. Some of our users feel very strongly about plaintext or HTML (and there are strong proponents in both camps), and removing the option will backfire in user satisfaction. This is a particularly emotional topic for some group of people, and I think it's worth to keep that one dropdown in the UI.
• In today's world, it makes sense to remove the per-mail "Ask me" dialog and default it to Both, unless the user made a difference choice in the preferences.

I mostly agree with what Ben wrote, however, why can we rethink the UI a bit altogether and make the system more functional?

For a message composed in the HTML editor, there are two options 1) Can be downgraded to plain text without loss 2) Cannot be downgraded.
So the first option the UI should offer is:
[ ] Send messages as plain text if possible.
That's already implemented.

The next issue comes from the address book: There the preference is unknown, plaintext, HTML. I assume that most contacts are in status unknown, so TB needs to use the option from the dropdown.
Instead I suggest this:
Change the address book to a binary choice: User must receive plaintext (which means they can also get plaintext+HTML).
Implement a true send option:
[ ] Always send as HTML-only
[x] Send as HTML-only unless a recipient requires plain text, in that case send plain text and HTML
[ ] Always send as plain text and HTML
The default should be the 2nd or 3rd option.
This approach guarantees that there aren't any surprises since the HTML is always sent, and it makes the plain text recipients happy. It also makes senders happy who prefer not to send out a larger message knowing that most recipients will not display the plain text part.

Just to clarify the previous behaviour for the discussion: the "mail.default_html_action" preference is only consulted if:

1. The user writes their messages in HTML (else it will always be in PlainText only).
2. It cannot be converted to plain text (else it will only be in PlainText).
3. The preference of the recipients is mixed between HTML or PlainText, or at least one of the recipients is not in the addressbook or has an unknown preference (else it will match the common preference).

The current patches don't change this. It's just that the "Ask" option has been removed for "mail.default_html_action", and it is now a hidden preference.

(In reply to Ben Bucksch (:BenB) from comment #13)

• I would keep the general "Plaintext vs. HTML-only vs. both" preference in the UI. Some of our users feel very strongly about plaintext or HTML (and there are strong proponents in both camps), and removing the option will backfire in user satisfaction. This is a particularly emotional topic for some group of people, and I think it's worth to keep that one dropdown in the UI.

I'm not sure why it would still be used for a new user. If a user only wants to send out only plain text, they can deselect "Compose messages in HTML format" in their account settings.

For users that compose in HTML, I'm not sure why sending plain text alongside their HTML would be a bad thing. You would only loose the HTML entirely if you set "mail.default_html_action" to "PlainText" in the past with the UI, and now can't change it back with the UI. But I'm not sure why a user would have chosen this option, whilst still composing in HTML.

Actually, it might be safer to only allow "mail.default_html_action" to be "HTML" or "Both". I feel like a "PlainText" value would only cause problems for a user if it were ever set. If they have deselecting "Compose messages in HTML format" in their account settings, then "mail.default_html_action" will never be used. If they haven't, then they would be using the HTML editor, with all the formatting options on display, but would loose all formatting in most cases!

@magnus Do you think I should also remove and ignore the "PlainText" option for "mail.default_html_action" as part of the migration?

Here's a screenshot of the current TB 91 Send Options dialog which we are discussing here as to how much of that can be removed or reduced or morphed into something simpler.

If a user only wants to send out only plain text, they can deselect "Compose messages in HTML format" in their account settings.

The plaintext composer does something very different from the sending format. It changes the composer UI and behavior to that of (essentially) notepad.exe. In contrast, the HTML composer can do much easier interleaving quotes, and has many other features that the plaintext editor does not have. Some section of users want to send only plaintext, even though they like the comfortable editor features. To use an illustration: Even if source code is saved as text files, you still don't edit the source code in notepad.exe, but you use an editor that helps you. So, this configuration (HTML/rich/featureful composer, but sending plaintext) gave users that, and they want to keep it, even if they are sending plaintext.

I feel like a "PlainText" value would only cause problems for a user

Users who choose this option know exactly what they are doing. Remember that this is the classic flamewar in email. Everybody has a slightly different opinion on it, which makes sense to them, and is logical to them, and that's fine. That's why we need the preference. It's easy to underestimate how strong and diverse the opinions are on this point.

The current configuration (including having this one sending format preference) works very well for a large array of our users with different opinions.

(In reply to Henry Wilkes [:henry] from comment #15)

Just to clarify the previous behaviour for the discussion: the "mail.default_html_action" preference is only consulted if:
3. The preference of the recipients is mixed between HTML or PlainText, or at least one of the recipients is not in the addressbook or has an unknown preference (else it will match the common preference).

That's about always since most address book entries will have an unknown preference. Hence my suggestion to change that and only allow die-hard plain text people to be identified.

For users that compose in HTML, I'm not sure why sending plain text alongside their HTML would be a bad thing.

It's unnecessary and wastes space. Any comment on the suggestions from comment #14?

BTW, how do you ensure that a message goes out in HTML-only in the current state? Mark everyone in the AB as HTML recipient? Or set mail.default_html_action to HTML-only if everyone in the AB has an unknown preference?

(In reply to newsfan from comment #14)

I mostly agree with what Ben wrote, however, why can we rethink the UI a bit altogether and make the system more functional?

For a message composed in the HTML editor, there are two options 1) Can be downgraded to plain text without loss 2) Cannot be downgraded.
So the first option the UI should offer is:
[ ] Send messages as plain text if possible.
That's already implemented.

FYI: the UI for this was removed as part of the first patch. The preference is still used, but hidden.

The next issue comes from the address book: There the preference is unknown, plaintext, HTML. I assume that most contacts are in status unknown, so TB needs to use the option from the dropdown.
Instead I suggest this:
Change the address book to a binary choice: User must receive plaintext (which means they can also get plaintext+HTML).

We could at least change the default for new contacts to be HTML, but I think that would be for a different bug.

Implement a true send option:
[ ] Always send as HTML-only

I don't think this option would be a good idea because we would be explicitly ignoring that a contact was entered as preferring plain text. You can do this currently if you set "mail.default_html_action" to "HTML" and never set a contact's preference.

[x] Send as HTML-only unless a recipient requires plain text, in that case send plain text and HTML

It's true that there is no way to do this currently unless you set all the contacts to explicit HTML or PlainText and the "mail.default_html_action" is set to "Both".

[ ] Always send as plain text and HTML

This is the current default behaviour, provided you never set a contact's preference.

In general, it does seem like "mail.default_html_action" isn't that useful anymore since anything besides "Both" seems like a bad idea...

I don't think this option would be a good idea because we would be explicitly ignoring that a contact was entered as preferring plain text.

That happens today if you mix a die-hard plain text person with an unknown person (I've just tried). The message goes out as HTML since that's the only way to guarantee HTML-only delivery. But I agree, that option (and the entire current behavior) isn't a good idea.

Hence my proposal: Let the user have a firm HTML choice almost independent of the recipients, only with the option to let die-hard plain text recipients have their plain text (which, I repeat, they are not getting today).

(In reply to newsfan from comment #20)

That happens today if you mix a die-hard plain text person with an unknown person (I've just tried). The message goes out as HTML

In this case, it should be going out as whatever "mail.default_html_action" is set to, which is HTML+PlainText by default. Have you set "mail.default_html_action" to 2 (HTML-only)?

Can we please keep this focused?

• The bug here is to remove the recipient-centric options - both domains and individual contacts. That I agree with.
• Likewise, it's a good idea to remove the mail-specific "Ask" dialog. I wrote that dialog back then, and I nonetheless agree with its removal.
• But please leave the sender-centric options exactly as they are. Some of our users have a very clear idea - and very strongly held opinions - how Thunderbird should send mail. The sender-centric options and defaults should not change.
  Changing the sender-centric options, or even just proposing to change that, is going to end up in a big flamewar. I'm just trying to put the flames out. The purpose of this bug was reasonable, and we should not go beyond that.

Looks like I'm repeating myself: Unless you set all AB entries to "prefer HTML" there is not way to guarantee HTML-only delivery (of which I'm a fan since it saves space). So yes, I have the pref set to HTML-only.
And repeating once again: There should be clear choice for the user to determine how they want to send "rich content", either as HTML-only or plain+HTML, only altered by the system if a die-hard plain recipient gets in the way.

anything besides "Both" seems like a bad idea...

I disagree. "Both" is a bad idea since it's unnecessary. I only know one version who requires plain text, and that person regularly gets HTML by accident (... since <repeat>there is not way to guarantee HTML-only delivery</repeat>).

We could at least change the default for new contacts to be HTML, but I think that would be for a different bug.

Too late for that, there is no way to mass change all ABs.

As Thomas wrote in comment #16: The whole thing should be "morphed into something simpler", clearer and more functional.

newsfan wrote in comment 23:

"Both" is a bad idea.

Please do not state your personal opinion as fact.
"Both" is our default for good reason. It's the most compatible mode, and works with almost every recipient.
I know many who have HTML mail reading disabled. You will also be downgraded by some spam filters, if you send HTML only.

@benb and @newsfan

To try and address both of your concerns. What would you think of:

• A new preference on whether to treat an unknown recipient as PlainText, HTML or HTML+PlainText (currently, an unknown recipient will act as if it is a mixed recipient list), which defaults to the latter.
• And if the recipient list is mixed between HTML and PlainText, we always send HTML+PlainText (this is where "mail.default_html_action" would normally come in, but we now ignore it).

If you want always only HTML, then you set the new preference to HTML and never set the contact's preference. Similarly if you want always PlainText or HTML+PlainText. So the new preference would work like "mail.default_html_action" did in these cases. But for users that actually set their contact's preferences, we will properly take them into account, and it will work better than currently. E.g. this would support the "Send as HTML-only unless a recipient requires plain text, in that case send plain text and HTML" option you suggested.

On this topic, have we got any bugs or support pages where any of these issues have come up?

(In reply to Ben Bucksch (:BenB) from comment #24)

"Both" is a bad idea.

Please do not state your personal opinion as fact.
"Both" is our default for good reason. It's the most compatible mode, and works with almost every recipient.
I know many who have HTML mail reading disabled. You will also be downgraded by some spam filters, if you send HTML only.

I think you misread my comment, I said (Edit: Nevermind, I thought your were replying to me. Sorry!)

anything besides "Both" seems like a bad idea

I.e. that "Both" is currently the only safe option for "mail.default_html_action" because any other value can end up ignoring an explicitly set contact PlainText preference. E.g., if you send a message to contact A, who prefers HTML, and contact B, who prefers PlainText, then we currently use "mail.default_html_action" even though we know that one of them won't receive the format they prefer and HTML+PlainText is likely the best choice.

Henry, I think Ben was referring to newsfan's comment 23 where they wrote

I disagree. "Both" is a bad idea since...

Re comment 25:
Henry, if you were to:

• remove the recipient-centric options (in address book, and domain based in preferences UI), and
• remove the mail-centric "Ask" dialog, and
• leave the sender-centric options completely alone
  • only changing the meaning of current 0 = Ask to now 0 = Auto = Both, and
  • moving the UI pref from the sub-pref-dialog into the Compose pref pane

then it should work as expected. Users can still set their preference of how they want to send mail (e.g. newsfan sets it to HTML-only, and others set it to plaintext-only, and the rest keeps the default = Auto = Both). This should not upset many people. However, altering or removing the sender-centric settings will upset lots of people.

Re comment 26 and 27:
Yes, aleca is right. I just retrospectively added an attribution line for clarity. Sorry that my quote was unclear.

(In reply to Ben Bucksch (:BenB) from comment #28)

Re comment 25:
Henry, if you were to:

• remove the recipient-centric options (in address book, and domain based in preferences UI), and

I thought we were keeping the address book contact options for a while still. But if we're removing them soon, that would simplify the work. I made my suggestion in comment 25 based on having to respect the contact options.

• remove the mail-centric "Ask" dialog, and
• leave the sender-centric options completely alone
  • only changing the meaning of current 0 = Ask to now 0 = Auto = Both, and

If we're removing address book contacts, then this would basically just be an initial default value for the compose window's "Options -> Delivery Format". We could also probably drop "Auto" since without the recipient checks it would just be "Auto-downgrade => PlainText, else HTML+PlainText" (https://searchfox.org/comm-central/rev/9ebb9d17182a3fef3b11a30fcbec14db9a2949fe/mailnews/compose/src/nsMsgCompose.cpp#4592).

I thought we were keeping the address book contact options for a while still.

Oh, I thought you wanted to remove them. That's how I understood this bug. I'd be OK with that.
But maybe I misunderstood.

We could also probably drop "Auto"

I'm in favor as keeping the pref value 0 = Auto as default for the preference that we keep. Because a) the preferences system deletes user preferences that match the default, and b) we have or had additional logic here, or might add other logic later, and c) it avoids a migration.

Interesting discussion here :-)
To mitigate HTML-only being sent to plaintext recipients due to the mix with "unknown" recipients we whipped up this tiny patch:
https://github.com/Betterbird/thunderbird-patches/blob/main/91/features/06-feature-override-html-only.patch

As Magnus wrote in comment #0, "cannot receive HTML email ... [...] is simply not a thing in 2022". Therefore the AB should not make the user have to think about what the "normal" correspondent will receive. The suggestion from comment #25 to add yet another preference defining how "unknown" should be interpreted is adding another layer of complexity (and was rejected before, see bug 1222176). A simple checkbox "make sure this guy/girl gets the plaintext (part!) they want" should suffice (which has a simple migration).
As for "have we got any bugs [...] where any of these issues have come up": Yes, plenty of bugs where due to misconfiguration everything was stripped down to plaintext, bug 1510134 and friends. There are heaps more, mostly linked to meta bug 889315. (You will have noticed by now, this is one of the most hotly contested areas in Thunderbird.)

(In reply to Henry Wilkes [:henry] from comment #25)

@benb and @newsfan

Thanks Henry for taking my concerns into account. As per bug 1222176 the first suggestion is not viable. If I understand comment #31 correctly, the patch presented there goes into the direction of your second suggestion, that is sending "both" in more cases and hence not strictly adhering to pref mail.default_html_action.

(In reply to Ben Bucksch (:BenB) from comment #17)

The current configuration (including having this one sending format preference) works very well for a large array of our users with different opinions.

Given the long trajectory of damage done with the current solution and the various attempts to fix it, this doesn't appear to be the case. Here's a summary of what needs to be addressed IMHO:

• Users with unknown preference are an outdated concept. Everyone should be able to accept HTML, unless especially marked (as die-hard text recipient). It is unfortunate that "unknow" is the default and that this default, unbeknownst to most users, then triggers the potentially fatal "default action" (see next dot point). It is also unfortunate that users generally wishing to send HTML-only can only do this via the (defective (see next dot point)) default mechanism or manually set the entire address book to HTML (and remember to do that with every auto-added new correspondent, an impossible task, really).
• Pref mail.default_html_action is dangerous as many bugs show: Either users select "plain" and then complain that their pictures got stripped since it's almost guaranteed that a user with unknown preferences is amongst the recipients and triggers that behavior. Or they choose "HTML" without realizing that plaint text recipients will be unhappy. Meanwhile I've had contact with the Mutt user: Yes, they can display HTML messages, but it inconveniences their workflow.

(In reply to Henry Wilkes [:henry] from comment #15)

@magnus Do you think I should also remove and ignore the "PlainText" option for "mail.default_html_action" as part of the migration?

No let's keep it.

Regarding "both" as default - from a purist pov maybe it's redundant, but from a storage saving perspective it's really hard to take seriously. You'd have to write until your fingers bleed to get the plain text up to megabytes, on a yearly basis. Then you send someone a picture the next day, and all the savings is down the drain...

I would not base discussion on the assumption that per-card pref of prefers plaintext would stay, since it is something that I think should be removed as well. If there is desire for a hardcore-plaintext-delivery mode, this could be an ample opportunity for add-ons to explore. An add-on could then set the per message delivery option based on whatever criteria it sees fit.

Magnus wrote in comment 34:

Regarding "both" ... from a storage saving perspective it's [neglegible] You'd have to write until your fingers bleed to get
the plain text up to megabytes, on a yearly basis. Then you send someone a picture the next day, and
all the savings is down the drain...

+1 Exactly that, particularly the last sentence.

I would not base discussion on the assumption that per-card pref of prefers plaintext would stay,
since it is something that I think should be removed as well.

Agreed. For the sake of this discussion, we should assume that all recipient-centric options are removed.

An add-on could then set the per message delivery option based on whatever criteria it sees fit.

Nice idea. I think adding an add-on API for that would be far easier than mainining the current complex code with all its corners.

(In reply to Magnus Melin [:mkmelin] from comment #34)

I would not base discussion on the assumption that per-card pref of prefers plaintext would stay,
since it is something that I think should be removed as well.

Do you have a design for the final solution? What is that final solution? If you remove the recipient's preference from the AB you are left with this:

• Most UI for setting mail.default_html_action (alongside the domain preferences) removed in this bug
• Recipient's preference removed from the AB (some other bug).

That leads to the final result:

• User can set send format on the compose window per message.
• mail.default_html_action will be applied to all HTML messages which don't have an individual choice, so users can choose to send all HTML messages as plaintext, even if not downgradable without losses. That doesn't make much sense.
• Add-on can set send format on the compose window per message based on whichever criteria, for example examining the recipients. Unclear which API is provided to set add-on-defined AB card properties for that purpose.

newsfan wrote in comment 36:

Most UI for setting mail.default_html_action (alongside the domain preferences) removed in this bug

That was not the purpose of this bug. The removal is probably accidential. I have objected to that here and in the review, and I am confident that Henry will retain the preference UI for the sender-centric preferences UI, including the UI for mail.default_html_action.

all ... plaintext ... doesn't make much sense.

It makes no sense to you, who is advocating for HTML, but it makes sense to some of our users. This is a highly controversial issue which almost always leads to flamewars, and I am asking you again for the 3rd time to not expand the discussion beyond the scope of this very specific bug.

I asked for the overall design, and you marked comment #36 "off-topic"? Please undo that. Generally in software development you know where you want to get to, even if you split the work into multiple steps (or bugs, like Mozilla does).

Composing in HTML and sending everything as plaintext is indeed very doubtful, to the point where Henry asked about removing that option.

Here is what I see as the final result:

• Recipient-centric:
  • Domain-based recipient-centric options are removed from preferences UI and logic. (this bug)
  • Contact-based recipient-centric options (checkbox) are removed from address book UI and logic. (this bug? For Henry and Magnus to decide.)
• Email-centric:
  • Per-email override in Composer menu - stays as is
  • Per-email "Ask" dialog is removed (this bug)
• Sender-centric:
  • Option: If email can be downconverted to plaintext, do that - yes/no, boolean preference, with UI, default true - stays as is
  • Otherwise: Send as plaintext, or HTML-only, or Both = HTML and plaintext - mail.default_html_action preference, with UI, default Both - stays as is, just the UI moves from the sub-dialog to the Composer prefs pane, see review remarks.

Scenarios:

• Default is to send plaintext when possible (just text etc.), otherwise both HTML and plaintext - doesn't change
• newsfan would "downconvert to plaintext" to false, and default_html_action to HTML-only, and always send HTML - works better than now, because bugs in edge cases like comment 31 are fixed
• Die-hard plaintext users would set "downconvert to plaintext" to true, and default_html_action to plaintext.

I think I will definitely remove the current UI for mail.default_html_action in this bug. The current UI describes it as

When sending messages in HTML format and one or more recipients are not listed as being able to receive HTML [use the preference format].

However, the way it is actually used (in nsMsgCompose::DetermineHTMLAction) is more like

When sending a message composed in HTML and there is a recipient with an unknown formatting preference, or there are two recipients with different formatting preferences, use the preference format (which will be used for all recipients).

So the preference is used in a different way to advertised, and neither implies the other. This is probably the cause for some of the linked bugs.

In the common case where a user never sets a contact's format preference, this reduces to

When sending a message composed in HTML, use the preference format.

This is more obvious as a kind of "Default send format", but I don't see how this can be inferred from the UI description, since it is outside its scope. My guess is that a user setting up a new profile would only use the preference like this if they had developer knowledge or were tipped off to do so.

I also think it is weird that this is a global preference, rather than per-account like "Compose messages in HTML format".

Personally, I would like to remove mail.default_html_action entirely to have a clean break with this disconnect. I feel bad for anyone who chose "When sending messages in HTML format and one or more recipients are not listed as being able to receive HTML: Convert the message to plain text" because almost all of their messages are being (lossy) downgraded.

But if mail.default_html_action has been a popular preference shared by people on forums who want to send as HTML or PlainText only (modulo recipient preferences) then we may have to keep it for the time being, and continue to help those who selected "Convert the message to plain text" to fix their situation.

One thing we could do to avoid the worst aspects of HTML -> PlainText downgrade is to ignore the "plain" value when the message's convertibility is nsIMsgCompConvertible.No, and send it as both instead. I.e. if a user slips up and uses a formatting option that would lead to a bad data loss, then we step in and send the html as well.

(In reply to Ben Bucksch (:BenB) from comment #39)

• Die-hard plaintext users would set "downconvert to plaintext" to true, and default_html_action to plaintext.

Sorry, that misses the point. Die-hard plaintext users/senders compose in plain text (see account options). People who compose in HTML usually want their HTML to be transmitted (unless downgradable), unless they know that there is a die-hard plaintext user at the other end who at least needs a plain text part.

Please clearly state a use case were a user composing in HTML would force their composition to all recipients to plain text all the time.

@Ben: I'd appreciate if you kept the discussion to BMO (as per the guidelines) and refrained from threatening personal messages as Thunderbird Council member just because someone questions your solution, which BTW, is not what the developer is intending if I understand comment #40 correctly. In comment #36 I asked about the overall design since this bug here cannot be solely judged by a yet not even clearly defined specific change in this bug.

This entire discussion started on TB planning, so how about to write up the design and present it there? This is not a bug fixing a crash or memory leak. You're starting to rip out parts of the UI and change the way the system has worked for years, badly, I may add, as the trajectory of bugs shows. So IHMO you owe it to the community, particularly as TB Council member, to present the final solution before creating a fait accompli.

I think I will definitely remove the current UI for mail.default_html_action in this bug.

I am objecting to that. We need this preference UI (of course moved to a new place). Some of our users care a lot and have very strong opinions about this (in either direction), and many will be very upset to have this option gone. We cannot live without UI for that preference.

The current UI describes it as ...

Yes, as I said, the intro label of course needs to change. I think the text was wrong anyways, even before this change. If you are lacking appropriate translated strings, you could completely remove the intro text. If really necessary, the dropdown options alone are sufficient.

Personally, I would like to remove mail.default_html_action entirely

We would massively lose users, in the following years. Because exactly our strongest advocates, the people who advocate for Thunderbird to be used in their families, friends, companies, they care a lot about this specific preference. We need the pref, with UI.

I also think it is weird that this is a global preference, rather than per-account like "Compose messages in HTML format".

The latter should probably be global as well (but it's irrelevant to this bug). Nobody wants to use 2 different text editors with completely different behaviors, in the same program. IIRC, the composer pref was originally made per-account, because we wanted to default newsgroup accounts to plaintext editor. But that makes little sense today. FWIW, almost nobody, not even die-hard plaintext fans, use the plaintext editor. Pretty much everybody uses the HTML editor, even for sending plaintext.

Here are the use cases (requirements) as I see them, feel free to add more:

1. User writes in plain text. They set this in the a/c manager. Plain text is sent. -- Working now.
2. User writes in HTML prefers to send out plain text if possible. -- Working now.
3. User writes in HTML wants to always send out plain text, even when lossy. Unclear whether this case exists, but mentioned in comment #42. Any telemetry on this?
4. User writes in HTML wants to always send out HTML-only text, also in combination with 2.
5. User writes in HTML wants to always send out HTML-only text, unless there is a plaintext recipient.
6. User writes in HTML wants to always send out "Both", for reasons mentioned previously (SPAM-detection, compatibility, no huge space saving).
7. User writes in HTML wants to set format individually.
8. User doesn't know, doesn't care and just wants to operate at default values or correct default behavior.

As long as you maintain the two prefs "plain text if possible" (mailnews.sendformat.auto_downgrade) and "HTML action" (mail.default_html_action) you can support all cases, except case 5 which needs to identify those plain text recipients. Whether the mail.default_html_action is kept in the UI depends on whether you want to cater for cases 3., 4. and 5. in the UI. If left in the UI, a radio button wording could be:
How to send HTML messages:
( ) Always downgrade to plain text (not recommended, will cause data loss for messages with rich content)
( ) Always send HTML only (may be inconvenient for recipients using text-based clients, may cause higher SPAM score)
(*) Always send plain text and HTML (recommended: Quasi-standard, best compatibility)

BTW, Outlook either sends plain text or "Both", it doesn't send HTML-only or do any auto-downgrade. In plain text mode OL doesn't permit formatting, however, it permits changing from HTML to plain text which causes the formatting to be lost, no * or / or other substitutions made.

newsfan wrote in comment 41:

Die-hard plaintext users/senders compose in plain text

FYI: No, they do not, see comment 17. To my knowledge, very few people use the plaintext composer. Even die-hard plaintext senders use the HTML composer.

The HTML composer is specifically designed to send perfect plaintext, but make the composition comfortable.

In my plaintext accounts I always use the plain text editor as html composer does not give any benefits in this case. Please get some numbers on the usage of different composer options and do not perform any changes based on every ones knowledge.

I'm proposing to not remove anything, aside from what this bug's title says.

I would consider the use of the HTML editor for sending only plain text an unsafe case that should not be exposed to the user. Thunderbird does not properly support this: it does not adjust the editor to only expose "safe" formatting options; there are no safety mechanisms in place when you use a formatting option or an insertion that will be lost; and there is no mechanism to preview the plain text just before sending. Composing in plain text is the only safe and supported way to send only plain text.

Moreover, there is no current UI that tells the user "select this option to convert all your messages to plain text by default". Personally, I only send plain text messages, but I've always used the "Compose messages in HTML format" option and the no-formatting editor. I doubt that a user would see "When sending messages in HTML format and one or more recipients are not listed as being able to receive HTML" and think they could use this to convert all their messages to plain text. To reiterate, to knowingly use this option to set a default sending format would likely involve a tip-off (on a forum, mailing list, etc) or reading the code. So for the few users that have been using this, and want to continue using the auto-downgrade with no safety mechanisms for data loss, I don't think a hidden preference will be outside their comfort zone. And on the flip-side, keeping it hidden means we are no longer advertising an option to users that has no proper support.

Regarding the HTML-only option. I don't think there are many problems with sending both HTML and plain text. It may be inelegant or in-principle wasteful, but it is a safe option. Therefore, I would consider the HTML-only option something only technical or niche users would require. So I similarly think there's no need to expose it in the UI, so it can remain hidden.

If it turns out this preference is more used than expected, it would need to be a fresh UI anyway because of the problems with the current wording. So we don't need to worry about loosing translated strings.

So overall, I think it is low risk to remove it.

Regarding contact preferences, if we are expecting an add-on to step in and handle this, we may need to delay its end of life before we wipe all the existing contact format preference data which users may have collected over time. Plus, it involves another component -- the addressbook -- and even the new addressbook still exposes this option quite prominently. Regardless of the details, I have a feeling removing them entirely will be delayed, so whilst I'm making changes in this section, I'll put in place a low maintenance fix to the ongoing problems that nsMsgCompose::DetermineHTMLAction has. Basically:

• If mail.default_html_action is set to "both", we always send both (we never look at the recipient preferences). This should be safe since all recipients get both versions, and their clients can select appropriately. Note, this would address removing the recipient-centric logic for most users.
• Otherwise, we only send the mail.default_html_action format if it does not conflict with an explicit contact preference. This can later be handed over to an add-on, which could do something more complicated.

(In reply to Henry Wilkes [:henry] from comment #47)
Sounds good!

Regarding address book: address book data is finally moving to really storing vCard data (work is just about to start).
The card UI for selecting format can be removed from the new address book. The relevant data will still be there, and also in the future as a vCard X-MOZILLA-HTML custom field, like it is today when exporting a card to vCard. So, there would be a way to see/add/modify this property and add-ons could make use of it - but really only to be used by people particular about that special case.

By default, we always send both HTML and PlainText.

Otherwise, if the user has opted to send just HTML or just PlainText by default, we only do so if there is no recipient preference that explicitly prefers a different format. Otherwise, we send both.

Moreover, if the composed message is not deemed convertible to PlainText without a hard data loss, we also send both.

Depends on D136009

The changes here look good, however, Ben may not me happy since you're removing the lossy forced downgrade.

Any reason for the i < MAX_OF_RECIPIENT_ARRAY && action != nsIMsgCompSendFormat::Both in the while loop? You break the loop after setting "Both". An early return might be preferable or wrapping the loop into an if-clause. Just a taste issue or am I missing something?

Henry wrote in comment 47:

there are no safety mechanisms in place when you use a formatting option or an insertion that will be lost

Actually, there are: It's the "Ask" dialog, which you are removing here. So, what you are really saying is that you should not remove this Ask dialog and preference option. So, given this argument, I'm actually going to say that the "Ask" dialog should not be removed.

I understand that you're talking about people who set the pref to "always plaintext". That value is for people who have seen the warning, and made their mind up, and do not use blue background and Helvetica 96px font. Users who set the preference to send 'only plaintext no matter what', they know full well not to use formatting. But they need the HTML composer for its other comfort, like quote handling.

I would consider the use of the HTML editor for sending only plain text an unsafe case that should not be exposed to the user.

The HTML composer is designed specifically for sending perfect plaintext. "HTML composer" is a actually mis-nomer, because the HTML composer is actually our primary plaintext composer. In fact, the HTML composer is better for sending plaintext than the plaintext editor, for reasons that are far far too complicated to explain in a single posting, but it would be a chapter of a book, or even an entire book to explain it all. We have a lot of code - all over the application and even in Gecko - to optimize this particular code path (HTML composer sending plaintext). Even all the way back to the mail viewer, so that the recipient can see the plaintext email perfectly, with wrapping line breaks, even within multiple nested quotes, and all that. Our plaintext handling works as designed only with the rich text composer.

This concerns wrapping line breaks, clearly designated and machine-readable quotes, nested quotes, and many other details in our plaintext handling that make email reliable. What's nice in Thunderbird is that the user doesn't need to worry about the details of formatting plaintext properly, like proper quotes, but we take care of creating the proper plaintext email, whereas the user uses a comfortable editor.

You've cited Outlook's behavior before. We are far superior to Outlook here. Our users specifically choose to not use Outlook, and use Thunderbird, in part exactly because our plain text handling is far superiour to Outlook. It's actually one of Thunderbird's core qualities. You're proposing to remove a core feature here that many users appreciate.

Thunderbird is popular as mail client, in part exactly because it has these options for sending perfect plaintext. Please leave these options for end users open, who want to use that.

You're touching settings here that are really core to Thunderbird, one of its main qualities, and options that many users feel extremely strongly about. HTML vs. plaintext is the classic flamewar in email. Taking choices from the users away is guaranteed to cause a huge backslash. And there is no need - these features and settings have proven to work very well.

I concur that the recipient-centric options can go away. I do not think that many users use them. However, I know that many of our biggest advocates and fans care deeply about the sender-specific options, particularly the mail.default_html_action, the "send plaintext when possible" settings, and their respective default values. Changing anything at all there will change all emails that a user sends out, will be noticed immediately, and will have a serious cost in losing mindshare and love from our users. You'd be enraging our userbase. I'm trying to say this the nicest way possible: Please don't do that.

Henry, just try this: Reply to a long email, and interleave (!) comments within the quote, at multiple points, i.e. reply to specific parts of the quote, and insert your reply in the middle of the quote. That's the "Internet" way to reply.

1. Outlook is not capable of that
2. Our plaintext editor allows it, but it's hard to do, because it doesn't re-wrap the quoted lines, so you can end up in very unfortunate linebreaks in your quote. Manually adjusting the linebreaks in the quote then becomes a major hassle. Worse, if you do it wrong, the software no longer recognizes it as quote, and the recipient sees raw ">" instead of a blue bar and wrapping quotes.
3. In our HTML composer, you continue to see nice vertical blue quote bars, instead of the raw ">", even while you are editing, same nice display as when you read mails. Our HTML composer then allows you to just place the cursor anywhere in the quote, hit return, and it breaks the quote into 2 halves at that point, and you can interleave the comment. When you do that, we can automatically re-wrap quotes, e.g. use linebreaks where appriopriate, because we can differentiate between hard and soft linebreaks in the original plaintext email. That is particularly helpful when there are multiple nested quotes, and that avoids the "comb" effect that Outlook creates when the quote level in plaintext is too deep.

And this is just about quoting, which is an critical everyday feature for many of our users. But there are many other details in the plaintext handling that only our HTML composer can do, which is why it's the best way to send plaintext emails.

I'd say we have hands-down the best plaintext sending and receiving capabilities - but only when using the HTML composer. That's why this combination is important.

Can we please not re-open this entire complicated topic every time we touch something?

(In reply to Ben Bucksch (:BenB) from comment #51)

Henry wrote in comment 47:

there are no safety mechanisms in place when you use a formatting option or an insertion that will be lost

Actually, there are: It's the "Ask" dialog, which you are removing here. So, what you are really saying is that you should not remove this Ask dialog and preference option. So, given this argument, I'm actually going to say that the "Ask" dialog should not be removed.

The "ask" dialog was only shown if the user set mail.default_html_action to "ask" (or for newsgroups). But the discussion here has been about users who have set mail.default_html_action to "plain". Such users would not be effected by the loss of the "ask" dialog, and never had a safety mechanism in place. Users that had set "ask", will now send both html and plain text unprompted.

(In reply to Henry Wilkes [:henry] from comment #53)

The "ask" dialog was only shown if the user set mail.default_html_action to "ask" (or for newsgroups). But the discussion here has been about users who have set mail.default_html_action to "plain". Such users would not be effected by the loss of the "ask" dialog, and never had a safety mechanism in place. Users that had set "ask", will now send both html and plain text unprompted.

I know. The safety measure was (originally) that the Ask dialog was the default, and users would have typically been going through that dialog. And then get annoyed about the dialog. And then set this preference, to always make this choice. But having seen the dialog, they would have been warned multiple times. That was the design. And then somebody went and didn't understand all that and changed the default from "Ask" to "Both". What we should have done instead was to keep the default at 0 = Ask, and add a "[ ] Remember this decision" checkbox in the dialog, which then sets the preference. So, users would see the warning once before making a choice.

Today, this is accessible only via preferences. Users who change this preference know exactly what they are doing. Hey, it's even in the word: "Plain text" implies that colors cannot show. So, when you explicitly and deliberately go into preferences and set this option, you know what you are doing and that you cannot make your text blue nor set the font.

Sorry for the slow review, it's a complex matter... Man, these preferences are even more intertwined than I recalled :(

Having looked through it again, it seems to me there's a strange dependency, or rather currently lack of dependency between the prefs. Thinking it through a bit more, I would say the (valid) behaviors are:

A) Prefers sending plain text: mailnews.sendformat.auto_downgrade true

• when convertible, no problem. send plain
• when convertible==No: should send both - anything else is dataloss (which we have bugs on). Most of the time, messages are convertible so should not be a biggy...
  ** For people really don't wanting to send html or both in any case, we have both the plain text composition AND the message centric delivery option to force plain text delivery

B) Does not prefer sending plain text: mailnews.sendformat.auto_downgrade false

• will send HTML or both, depending on pref. The user already preferred not to send plain text, so indeed, as Henry suggested, that is really not a very valid option.

newsfan falls into B), and gets HTML as default. For others - A), it's plain or both depending on the message

Makes sense?

(In reply to Magnus Melin [:mkmelin] from comment #55)

A) Prefers sending plain text: mailnews.sendformat.auto_downgrade true

• when convertible, no problem. send plain
• when convertible==No: should send both - anything else is dataloss (which we have bugs on).

This is what I did in https://phabricator.services.mozilla.com/D136245#C4677881NL4622 (not sure if this is a permanent link).

Most of the time, messages are convertible so should not be a biggy...

There is a caveat in that anything non-convertible in the quote reply will make the overall message non-convertible. Plus this covers a range of cases which includes embedded images, but also any set font family, size or color. Whether loosing these from a quote reply would be "bad" data loss is context dependent. In principle, a user could "Remove Text Styling" in the quote reply if they really want to. But more importantly, as you said

** For people really don't wanting to send html or both in any case, we have both the plain text composition AND the message centric delivery option to force plain text delivery

This is the supported way to compose plain text messages. If there really is a demand to have more "formatting" options for plain text, then those should be exposed in the plain text editor. E.g. pressing Bold inserts ** and places the cursor between them. Although I'm not sure there is this specific demand since it is very easy, and probably quicker, to type the *s yourself.

(In reply to Henry Wilkes [:henry] from comment #56)

There is a caveat in that anything non-convertible in the quote reply will make the overall message non-convertible.

Sure, but that is also the case today.

(In reply to Magnus Melin [:mkmelin] from comment #55)

newsfan falls into B), and gets HTML as default.

Actually, no. I prefer sending plain text, but if not convertible, HTML-only. Only if the recipient is a Mutt user, send "Both". See comment #43, case 5 plus 2. Looks like this is what Henry's changes will do, see comment #50.

Henry, the recommended and supported way to send plaintext messages in Thunderbird is to use the HTML composer and then let Thunderbird downconvert the message to plaintext during sending. Plaintext is far more complicated than you would think and only the HTML composer gives us sufficient information to send good plaintext. The plaintext composer does not. I know it's ironic and not logical superficially, but there is far far more depth to this question than one realizes at first. Additionally, the HTML composer is also far easier to use for most users. So, the combination of the HTML composer and plaintext downconvert is the better option both for ease of use, and for best plaintext results. This combination is the supported and recommended way to send plaintext, not the plaintext composer. The plaintext composer is mainly for people who want to send ASCII-Art or want to play with individual characters in specific ways, so there are some use cases, but it's not fit for most use cases.

Please see comment 51, where I explained in detail why the HTML composer is needed to send good plaintext. Explaining everything would take a whole chapter of a book.

(In reply to newsfan from comment #58)

Actually, no. I prefer sending plain text, but if not convertible, HTML-only. Only if the recipient is a Mutt user, send "Both". See comment #43, case 5 plus 2. Looks like this is what Henry's changes will do, see comment #50.

But recipient-centric will be gone, so there is no such Mutt user case.
What I propose is to - unless specified in per-message delivery by the sender - disregard your wish to send HTML-only when not convertible, and send Both instead. (Which, I'm sure the mutt user would prefer over HTML only.)

(In reply to Magnus Melin [:mkmelin] from comment #60)

But recipient-centric will be gone, so there is no such Mutt user case.

As far as I understood, the preference in the address book will not be removed in this bug. D136245 still makes reference to recipient.mCard->GetPropertyAsUint32(kPreferMailFormatProperty, &preferFormat);.

What I propose is to - unless specified in per-message delivery by the sender - disregard your wish to send HTML-only when not convertible, and send Both instead.

If I understand this correctly, the system will then send plain text if downgradable and "both" otherwise to all recipients hence ignoring the HTML-only choice of the sender. IMHO not a good idea. Unless a die-hard plain-text-only recipient is involved, there is never any point in sending "both": Either the e-mail is downgradable, then plain text is sufficient, or it is not downgradable, then sending a lossy plain text part makes little sense since 1) the part is defective by its nature and 2) no decent client will view it. And if TB is used as decent client by the recipient, it can display the HTML message as plain text if the recipient desires. In fact, that was the practice for years, only more recently TB now displays the plaintext part in a plain text view.

(Which, I'm sure the mutt user would prefer over HTML only.)

Correct, my Mutt correspondent can deal with "both" but complains about the inconvenience of viewing HTML-only. However, for the one Mutt correspondent I wouldn't want to send "both" to everyone (as per your suggestion), especially if the Mutt person is not involved at all.

(In reply to newsfan from comment #61)

(In reply to Magnus Melin [:mkmelin] from comment #60)

But recipient-centric will be gone, so there is no such Mutt user case.

As far as I understood, the preference in the address book will not be removed in this bug.

I think it would make this bug easier to move forward if we actually handle ignoring the address book pref as well in this bug. Building in assumptions that that pref would be honored in the future is just making things more difficult so discuss. It also seems not to be very controversial to do so.

If I understand this correctly, the system will then send plain text if downgradable and "both" otherwise to all recipients hence ignoring the HTML-only choice of the sender. IMHO not a good idea. Unless a die-hard plain-text-only recipient is involved, there is never any point in sending "both": Either the e-mail is downgradable, then plain text is sufficient, or it is not downgradable, then sending a lossy plain text part makes little sense since 1) the part is defective by its nature and 2) no decent client will view it.

Thunderbird generally produces better plain text version than other tools, so IF anyone will look at it in plain text, it's likely to be the best they will get.

Regarding 2), you're contradicting yourself - your Mutt user will want the plain text part. Can of course be argued whether Mutt is a decent client or not.

Correct, my Mutt correspondent can deal with "both" but complains about the inconvenience of viewing HTML-only. However, for the one Mutt correspondent I wouldn't want to send "both" to everyone (as per your suggestion), especially if the Mutt person is not involved at all.

Well, I suppose we could have mail.default_html_action take effect in the non-convertible case as well. BUT - limit it to html-only or both
Given that pref had prior entanglement, perhaps we should drop that pref completely, and instead have a new hidden one - a boolean of whether to send html-only or both, when we're not sending plain text.

(In reply to Magnus Melin [:mkmelin] from comment #62)

Thunderbird generally produces better plain text version than other tools, so IF anyone will look at it in plain text, it's likely to be the best they will get.
Regarding 2), you're contradicting yourself - your Mutt user will want the plain text part. Can of course be argued whether Mutt is a decent client or not.

There is no contradiction I can see:

• Mutt recipient wants a good plaintext part, agreed.
• TB recipient can downgrade HTML to plain text if they wish to view plain text with the same code used for downgrade for sending.
• Other recipients will (likely) ignore the plain text part and never view it.

TB recipient can downgrade HTML to plain text if they wish to view plain text with the same code used for downgrade for sending.

The HTML part may end up lost: e.g. mailing list software doing their own down-conversion when they didn't get a text part.

we actually [remove] the address book pref as well in this bug.

Agreed. I don't think many people use it extensively. It simplifies the logic, and it matches the bug title.

Thunderbird generally produces better plain text version than other tools, so IF anyone will look at it in plain text, it's likely to be the best they will get.

Exactly.

mail.default_html_action ... perhaps we should drop that pref completely

This pref is highly important for many of our users, and we need it in exactly the way it is today.

FWIW, this pref existed independent from the recipient-centric prefs we have today. The latter were added in retrospect, and the logic never worked well, and I think it's fine to remove the recipient-centric prefs, but let's not throw the baby out with the bath water. *

** throw the baby out with the bath water: To discard something valuable or important while disposing of something considered worthless

(In reply to Ben Bucksch (:BenB) from comment #65)

mail.default_html_action ... perhaps we should drop that pref completely

This pref is highly important for many of our users, and we need it in exactly the way it is today.

We're removing the ask-me-what-to-do dialog so it can't stay exactly the same.
It seems unlikely people would choose to ever silently allow data-loss. To the extent force-everything-plaintext people exist, they may have been using the ask-me option and then likely still chose to send Both... now that we won't have the dialog, we need figure out what to do there.

We're removing the ask-me-what-to-do dialog ... now that we won't have the dialog, we need figure out what to do there.

If that's a factor, then don't remove it.

The ask-me-what-to-do dialog is not related to the per-recipient options. In fact, the ask dialog existed years before the per-recipient address book option.

Tweaking summary for better readability and retrievability.

Per-recipient Prefers to receive messages formatted as: Unknown | Plain Text | HTML (PreferMailFormat AB card property) has been removed in bug 1758460.

After much time spent on discussing this issue (and related issues stemming from discussions around this bug) - the solution that I think we need to move on is the following:

• Remove recipient and domain centric prefs
• Leave one pref to send only plaintext or not

For those sending only plaintext:

• Update the compose editor to disable rich text (links, images, etc)
• Sanitize pasted or dragged text to remove links, images, etc.
• Disable all menu items that interact with those elements

This removes confusion around multiple prefs, we let the user decide if they need to only communicate through plaintext. The rich text editor remains and allows for a decent composing experience, but we disable all the rich content options and sanitize copied text (or content coming from a forward or replied email).

I think there should be some details of the implementation figured out so that a user is aware (reminded) that they are in plaintext mode once it is enabled. But I leave that up to Henry and Alex to sort out.

For those sending only plaintext:
Update the compose editor to disable rich text (links, images, etc)
Sanitize pasted or dragged text to remove links, images, etc.
Disable all menu items that interact with those elements

We have patches for that, and we'll contribute them - in a new bug ticket, to avoid having too much happening here in this bug.

(In reply to Ryan Lee Sipes from comment #69)

After much time spent on discussing this issue (and related issues stemming from discussions around this bug) - the solution that I think we need to move on is the following:

Thanks for taking everything into account and deciding on a direction.

• Leave one pref to send only plaintext or not

Can you clarify what this means? We currently have two preferences:

• mailnews.sendformat.auto_downgrade - Controls whether we send plain text if the message only contains plain paragraphs (and plain quotes).
• mail.default_html_action - Controls whether we send plain text, html, or both. Which has been the main focus of the bug discussion.

The first preference becomes redundant when the second is set to send only plain text. So there are only really 5 states:

1. Always only send plain text.
2. Always only send html.
3. Always send both html and plain text.
4. Send plain text if the message is plain, otherwise send both html and plain text.
5. Send plain text if the message is plain, otherwise send only html.

Which of these states are we keeping and exposing in the UI? Do we want to present them as two separate preferences as we do now, or merge them into one?

For those sending only plaintext:

• Update the compose editor to disable rich text (links, images, etc)
• Sanitize pasted or dragged text to remove links, images, etc.
• Disable all menu items that interact with those elements

We also need to ensure that the quoted message is similarly sanitized.

• Leave one pref to send only plaintext or not

Can you clarify what this means? We currently have two preferences:

• mailnews.sendformat.auto_downgrade - Controls whether we send plain text if the message only contains plain paragraphs (and plain quotes).
• mail.default_html_action - Controls whether we send plain text, html, or both. Which has been the main focus of the bug discussion.

The first preference becomes redundant when the second is set to send only plain text. So there are only really 5 states:

1. Always only send plain text.
2. Always only send html.
3. Always send both html and plain text.
4. Send plain text if the message is plain, otherwise send both html and plain text.
5. Send plain text if the message is plain, otherwise send only html.

Which of these states are we keeping and exposing in the UI? Do we want to present them as two separate preferences as we do now, or merge them into one?

We are going to use a checkbox (or some toggle) for "Send only in plain text." If that is not set, I think we should convert to plaintext if possible (which I think is what we do now), otherwise send both html and plaintext (option 4).

For those sending only plaintext:

• Update the compose editor to disable rich text (links, images, etc)
• Sanitize pasted or dragged text to remove links, images, etc.
• Disable all menu items that interact with those elements

We also need to ensure that the quoted message is similarly sanitized.

Agreed.

(In reply to Henry Wilkes [:henry] from comment #71)

1. Always only send plain text.
2. Always only send html.
3. Always send both html and plain text.
4. Send plain text if the message is plain, otherwise send both html and plain text.
5. Send plain text if the message is plain, otherwise send only html.

Which of these states are we keeping and exposing in the UI? Do we want to present them as two separate preferences as we do now, or merge them into one?

To add on top of Ryan's answer, we should only offer a single option to remove any confusion.
The option would be "Always only send plain text".
For any other scenario we will handle it automatically by detecting if we can send both plaintext and HTML, or if the message is pure text we can send plaintext.

The only option that users might care about is enforcing Plaintext no matter the scenario.
In the preferences, we need to be sure to properly inform the user (with description, tip-caption, etc) that if that option is enabled, the composer will be stripped from any rich text feature, and we will forcibly sanitize all content (expected data-loss).

Maybe we should also consider adding a more "clear and obvious" indicator in the composer when plaintext is enforced, with a handy link to that pref, in order to not exclusively rely on missing icons or disabled features to represent the plaintext state.

Send plain text if the message is plain, otherwise send both html and plain text.

FWIW, this is currently the default, and the default should not be changed.

There are some users who want to always send HTML. So, may I suggest:

• Always send HTML and plaintext
• Always send plaintext
• Automatic (default)

This could be a single dropdown. The "Automatic" option would send Send plain text if the message is plain, otherwise send both html and plain text.

In the backend, we would keep the two sender-centric preferences that we currently have:

• mailnews.sendformat.auto_downgrade - Send plain text, if the message only contains plain text - Default: true
• mail.default_html_action - Controls whether we send plain text, html, or both. - Default: both, unless previous option applied.

These are sender-centric, i.e. express our user's opinion and preference. As discussed, our users have very strong opinions on this.
The objective of this bug was to remove the recipient-centric options (where our user sets these format preferences for specific recipients only). Removing the recipient-centric option will allow us to get rid of most of the UI and the entire sub-dialog, and some confusing and buggy backend-logic.

With this, the code will be less complex, and the UI surface shrinks to a single dropdown.

(In reply to Ben Bucksch (:BenB) from comment #74)

Send plain text if the message is plain, otherwise send both html and plain text.

FWIW, this is currently the default, and the default should not be changed.

Agree.

There are some users who want to always send HTML. So, may I suggest:

• Always send HTML and plaintext
• Always send plaintext
• Automatic (default)

Yes, we can have show these options in the Settings > Compose area.

This could be a single dropdown. The "Automatic" option would send Send plain text if the message is plain, otherwise send both html and plain text.

I would use a radio group, which immediately shows the users the available options and allows us to add description and better explanatory text underneath each radio button.
I'm gonna do a quick mock-up.

(In reply to Henry Wilkes [:henry] from comment #71)

there are only really 5 states:

1. Always only send plain text.
2. Always only send html.
3. Always send both html and plain text.
4. Send plain text if the message is plain, otherwise send both html and plain text.
5. Send plain text if the message is plain, otherwise send only html.

Note, if we're ditching state 5, then the remaining options align with the 4 menu items in the compose window's "Options: Delivery Format" menu, with state 4 aligning with "Auto-detect". So we could use this preference to simply select one of the 4 by default when loading the compose window. We could drop a bunch of C++ code as well.

So it might make sense to keep all options 1 to 4, and make the wording in the preferences and the menu align.

In terms of migration, I think we could safely migrate state 5 to state 2 or state 4. Since state 5 is an odd configuration, I think probably translate it to the default state 4.

2. Always only send html.

I'm not against having 4 options instead of 3, but does this option really make sense?
Do we know the reasons why a user might not want to send also plaintext? Are these reasons valid to justify this extra option?
I don't have a strong opinion on this, I'm just asking as I'm curious about the usefulness of that option.

Consistency between the Compose menu options and Preferences is key, so we can align those. So whatever we decide, it should be identical in both scenarios.

Henry wrote:

we could use this preference to simply select one of the 4 by default when loading the compose window. We could drop a bunch of C++ code as well.

I agree it makes sense to initialize the compose window option with the general preference. I thought that was already the case. If not, we should definitely do that. I am not sure how much code that will save, though. At least the Automatic option needs code after sending.

we could safely migrate state 5 to state 2 or state 4

(I see that you modified your comment after posting. Thank you for that.)

This state 5 could migrate to state 4, indeed. (Not 2, though.)

Alex wrote:

I would use a radio group

+1

I'm not against having 4 options instead of 3, but does this option really make sense?
Do we know the reasons why a user might not want to send also plaintext? Are these reasons valid to justify this extra option?

There's no real technical argument to send only HTML, given that HTML-capable mail readers will just ignore the plaintext part, and non-HTML mail readers will use the plaintext part. The only downside is generally about 1 KB larger messages, which in this day and age is really neglient. A single image of 4 MB by email equates 4000 to 6000 plaintext emails.

I would then suggest to do the following:

• Hide option 2 (Always only send HTML) behind a pref.
• Add telemetry to track which option is the most used.
• Halfway through the 114 cycle we can check if we get any complaint and sensible requests for that option, and if not, we can drop it.

By doing so, we can temporarily leave the code handling that option in case we get a decent amount of requests to add it back (big doubt).
Meanwhile we gracefully reduce the amount of options and simplify this area, and if no complaints are raised, we drop it entirely around 104 or 105, without uplifting.

WOW. [...]

(In reply to Henry Wilkes [:henry] from comment #71)

there are only really 5 states:

1. Always only send plain text.
2. Always only send html.
3. Always send both html and plain text.
4. Send plain text if the message is plain, otherwise send both html and plain text.
5. Send plain text if the message is plain, otherwise send only html.

Thanks Henry for the correct and concise listing of the 5 possible resulting behaviours.
I propose to continue supporting all of them.

(In reply to Henry Wilkes [:henry] from comment #77)

Note, if we're ditching state 5, then the remaining options align with the 4 menu items in the compose window's "Options: Delivery Format" menu, with state 4 aligning with "Auto-detect". So we could use this preference to simply select one of the 4 by default when loading the compose window.

That's the basic idea of what I proposed 6 years ago in bug 1228846. Only at the time I was probably still struggling with the actual behaviour and usefulness of Send messages as plain text if possible, the checkbox for which was introduced following my advocacy to address significant user dissatisfaction with the destructive and unpredictable algorithm of auto-downgrade at the time (now mostly tamed).

I propose to keep state 5, which combines the best of both worlds and produces perfectly trimmed and simple messages.
I would even propose that as Thunderbird default.
More below.

I have significant doubts about offering users a choice to default to dataloss without warning with 1. Always only send plain text.
If Ben insists on supporting that option, I would consider not exposing it in the UI.
If we keep it in the UI, it needs a strong dataloss warning - and composition UI should also have some warning indication.

(In reply to Alessandro Castellani [:aleca] from comment #78)

2. Always only send html.

I'm not against having 4 options instead of 3, but does this option really make sense?

We currently have 4 options in the delivery format menu, so I think it's about having 5 or 4?

Iiuc, you're asking if option 2. Always only send html makes sense?
This option will make a lot of sense to many users, and we have ample evidence for that on bugs, the latest being bug 1759668 which Matt filed based on user support issues.
Especially enterprise will need a reliable wysiwyg option where Thunderbird does not alter their messages in any way compared to what they see in composition. HTML formatting has the highest chances of rendering as intended with the recipient. At the same time, this option skips the unnecessary and wasteful plaintext duplicate copy of each message, which also simplifies the message structure. All common, modern mail clients and webmail can handle HTML and do not need nor show the plain text part ever.

Do we know the reasons why a user might not want to send also plaintext? Are these reasons valid to justify this extra option?
I don't have a strong opinion on this, I'm just asking as I'm curious about the usefulness of that option.

Why should we force 30 million users to send double copies of their messages (HTML+plain) whereas the vast majority of recipients (99%?) will never ever look at the plain text part? That's wasteful and inefficient. Simple message structure needs less parsing, and is environmentally friendly (transmission and long-term running server space).

(In reply to Ben Bucksch (:BenB) from comment #79)

There's no real technical argument to send only HTML, given that HTML-capable mail readers will just ignore the plaintext part, and non-HTML mail readers will use the plaintext part.

Imho, sending cleaner and leaner messages is a good technical argument for sending only HTML.
Can you provide examples of modern and common non-HTML mail readers?
A non-HTML reader would render most of today's really circulating email useless, so such clients could serve only a very tiny fraction of users (1%?). Let's be honest, MUTT is an extreme niche case.

(In reply to Alessandro Castellani [:aleca] from comment #80)

I would then suggest to do the following:

• Hide option 2 (Always only send HTML) behind a pref.
• Add telemetry to track which option is the most used.

Not sure if I understand the plan - you want to add telemetry to check if an option which you are actively hiding is the most used?
That would give an unfair advantage to the visible options (self-fulfilling prophecy).
What's bad about sending HTML-only always so that we would need to hide that option?
Users will typically only choose between these 5 options in the prefs once in their life, so the number of options really won't matter much.
It's the option with the highest chances of rendering as intended, while avoiding the plaintext overhead.

Thanks Thomas for your input.

Let's try to keep this tight and short and not reiterate too much. We're almost reaching 100 comments and this bug doesn't really warrant that type of complexity.

This is what we're doing:

• Completely remove recipient and domain centric preferences.
• Move the Sending options in the Settings > Compose area, not inside an sub dialog.
• Offer only 4 options (we want to simplify things and bring some sanity for us the and the users)
  • Automatic (default)
    [description] Thunderbird will send HTML+Plaintext or Plaintext emails based on the content of the message.
  • Always send HTML and plaintext
    [description] Thunderbird will always send both version of the same message. The receiving application will handle which format will be shown.
  • Only send plaintext
    [description] Thunderbird will only support the plaintext format, disabling any rich text formatting in the message composer.
  • Only send HTML
    [description] Thunderbird will only support the HTML format. Applications that don't support HTML might have issues reading sent messages.
• The same 4 options will be available in the composer menubar for quick update if needed.

Let's stick with this for the first patch and initial solution.
If something is needed later on, we can improve it with follow ups, but let's try to keep the discussions and various examples (enterprise vs "regular" users) at minimum.

P.S.: The descriptive text is only an indication of what should be exposed to the user, as someone more versed than myself should write those

(In reply to Alessandro Castellani [:aleca] from comment #80)

I would then suggest to do the following:

• Hide option 2 (Always only send HTML) behind a pref.

That sounds like a bad idea also in view of Bug 136502 (6 duplicates), where many users are on record searching for a safe way to ensure HTML format for sending - I'm sure if you asked them they would happily go for Always HTML-only. The fact that we're defaulting to HTML+Plain says nothing about what the majority of users would really want to send if they fully understood the choices.

Bug 136502 - Provide prefs. setting to turn off Options > Delivery Format > Auto-Detect (especially to avoid Auto-Downgrade mystery/surprise)

(In reply to Alessandro Castellani [:aleca] from comment #83)

Thanks Thomas for your input.

+1 on maintaining the HTML-only option.

• Only send plaintext
  [description] Thunderbird will only support the plaintext format, disabling any rich text formatting in the message composer.
• The same 3 options will be available in the composer menubar for quick update if needed.

plaintext-only is already available as account option. If I understood Ben correctly, some users want to use the HTML editor and send plaintext, only offering the plaintext editor will not satisfy those users.

Additionally you're rolling part of bug 140800 (from 2002) where people asked to convert between HTML and plaintext conversion into this bug. By offering the three options together with removing the HTML editing, you will now have to implement this, with all the data-loss it may entail. To switch composition type on the fly is a (difficult) job far beyond the scope of this bug. Or is there a misunderstanding? Send options so far have always been about sending, never composing.

(In reply to Thomas D. (:thomas8) from comment #84)

(In reply to Alessandro Castellani [:aleca] from comment #80)

I would then suggest to do the following:

• Hide option 2 (Always only send HTML) behind a pref.

That sounds like a bad idea also in view of Bug 136502 (6 duplicates), where many users are on record searching for a safe way to ensure HTML format for sending - I'm sure if you asked them they would happily go for Always HTML-only. The fact that we're defaulting to HTML+Plain says nothing about what the majority of users would really want to send if they fully understood the choices.

Bug 136502 - Provide prefs. setting to turn off Options > Delivery Format > Auto-Detect (especially to avoid Auto-Downgrade mystery/surprise)

Okay, let's keep it for now and move forward. Update Comment 83 with the action plan.

(In reply to newsfan from comment #85)

plaintext-only is already available as account option. If I understood Ben correctly, some users want to use the HTML editor and send plaintext, only offering the plaintext editor will not satisfy those users.

We will always use the HTML editor even when the option is set to plaintext. We're doing that to ensure proper usability.
We will disable any rich text editing and sanitize pasted, quoted, forwarded text to guarantee a correct visualization of what the user has selected in the Preferences.

To switch composition type on the fly is a (difficult) job far beyond the scope of this bug. Or is there a misunderstanding? Send options so far have always been about sending, never composing.

We're focusing on the preferences for now, the changes in the compose options is just for visual consistency for now.
We will deal with those later in follow up patches.

(In reply to Ben Bucksch (:BenB) from comment #74)

There are some users who want to always send HTML. So, may I suggest:

• Ben, I understand from your above ^^ sentence that you wanted to include "always send HTML" in your proposal. I am failing to see which of your 3 options to choose for "always send HTML"?
• By extension, in your proposal, how can I send plaintext if possible (simple message), but otherwise HTML-only, which would seem like the best of both worlds, minimalist and resource-friendly?

• Always send HTML and plaintext
• Always send plaintext
• Automatic (default)

This could be a single dropdown. The "Automatic" option would send Send plain text if the message is plain, otherwise send both html and plain text.

Thanks for the update, but comment #83 is unclear. Can you please relate the four options to the original five?

1. Always only send plain text -> Only send plaintext
2. Always only send html.
3. Always send both html and plain text -> Always send HTML and plaintext
4. Send plain text if the message is plain, otherwise send both html and plain text -> Automatic
5. Send plain text if the message is plain, otherwise send only html.

"Only send HTML" is number 2 or number 5? If you don't offer 5 you'll deprive people of the "auto-downgrade" if possible. Why not offer all five options and equip them with telemetry for now?

Your answer in comment #87 is equally unclear. If you offer switching from plaintext delivery to HTML delivery in the compose window and you want to ensure "correct visualization", you'll need to implement a HTML to plaintext downgrade in an existing compose window which is one part of bug 140800. Say the user had "This is bold", you need to change this to This is *bold*, remove colors, images, etc. Just disabling the HTML editing controls is not sufficient. Note that in the past, changing HTML delivery to plaintext delivery had the effect that HTML editing was disabled. That function was removed explicitly in bug 1288914, because it didn't downgrade HTML content back then. Maybe you're planning on doing that now.

This is what Outlook does. It offers two (plus "rich") text formats and if you downgrade from HTML to plaintext with "rich" content, you'll get a warning. Obviously TB with five possible options is more sophisticated but currently doesn't offer the forced downgrade.

BTW, it is no surprise that we're nearing comment #100 here, since you're reviving 20 years of discussion on composition and delivery formats.

newsfan wrote in comment 90:

we're nearing comment #100 here

See comment 22. That comment was in response to you, newsfan.

Thomas D. wrote in comment 88:

Ben, ... I am failing to see which of your 3 options to choose for "always send HTML"?

With "always send HTML", I meant "HTML+plaintext". See comment 79, at the end.

Alex' comment 83:
+1 This is a good solution

(In reply to Thomas D. (:thomas8) from comment #2)

• Is Text+HTML still a format which we need to offer?

Thunderbird absolutely still needs to be able to send plain-text messages (at least when using MIME), as this is mandated by RFC 2049: Section 2 states that conforming mail clients "must be able to send at least text/plain messages". Apart from this, Section 5.1.4 of RFC 2046 sets out the motivations for including multiple formats of the same content, which I think are still very much relevant today. For instance, I might use Thunderbird to compose an HTML e-mail using some combination of colours and font sizes that the recipient may find annoying (due to personal preference) or even illegible (possibly due to sight impairment). Rather than simply assuming that the recipient is using a mail client that can strip the formatting (and that he or she knows how to use this feature), why not simply follow the long-established practice of including a plain-text version as part of a MIME multipart/alternative message?

Marking the last comment offtopic because no one is removing html+plaintext delivery.
Please, read Comment 83 before commenting.

ok, the latest version of the patches follow the plan in comment 83. They are ready for review again.

Still to do is to add the UI for setting the new "mail.default_send_format" preference, but that'll be in a later patch.

Depends on D136009

(In reply to Ben Bucksch (:BenB) from comment #70)

For those sending only plaintext:
Update the compose editor to disable rich text (links, images, etc)
Sanitize pasted or dragged text to remove links, images, etc.
Disable all menu items that interact with those elements

We have patches for that, and we'll contribute them - in a new bug ticket, to avoid having too much happening here in this bug.

Is this bug with patches coming soon? We would need something before 102 for the new text in the preferences to make sense.

Also, how do we plan on handling when a user selects "Options > Send Format > Only Plain Text" during composition? Are we going to similarly disable the styling elements? Also, what should we do with the existing message at this point? If we sanitise it we should warn the user if it will lead to data loss. We also need to consider what would happen if the user then presses Control+Z: just like that I suspect the xul:editor would reverse the sanitisation. If no one has a nice clean solution for this, I'll open a separate bug and we can just leave it as it was -- silently convert the message, with potential data loss, when we send it -- until we figure something out.

Thanks for the reminder, this had dropped from my radar. I filed bug 1766645 and attached the patches there.

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/10c851903cbb
Remove sendoptions dialog. r=mkmelin,BenB
https://hg.mozilla.org/comm-central/rev/5c4b637c1419
Remove plaintext_domains and html_domains preferences. r=mkmelin,BenB
https://hg.mozilla.org/comm-central/rev/9c466005fa68
Replace mail.default_html_action and mailnews.sendformat.auto_downgrade with mail.default_send_format. r=mkmelin,BenB
https://hg.mozilla.org/comm-central/rev/17aeba8cd394
Add UI for changing the mail.default_send_format preference. r=aleca,BenB

We think that keeping a HTML-only option with potential downgrade is worthwhile, and judging by comment #81 we're not alone:
https://github.com/Betterbird/thunderbird-patches/blob/main/102/features/11-feature-automatic-plain-or-html.patch

(In reply to Rachel Martin from comment #99)

We think that keeping a HTML-only option with potential downgrade is worthwhile, and judging by comment #81 we're not alone:
https://github.com/Betterbird/thunderbird-patches/blob/main/102/features/11-feature-automatic-plain-or-html.patch

FYI: if you make nsIMsgCompSendFormat.PlainOrHTML have the value 4 then any draft message or template saved before the patch will now be sent as plain text or html, rather than the user's preference. You probably want to keep Unset as the 4 value, and use 5 for your new enum. See the message in https://hg.mozilla.org/comm-central/rev/9c466005fa68 for more context.

Thanks for the advice.