Test case: https://searchfox.org/comm-central/source/calendar/test/browser/invitations/data/teams-meeting-invite.eml

The links show ok in the invite. However, after you accept this for a google caldav calendar, you can't see the description properly anymore. All the important information - the link by which you can join!! - is not visible anywhere, not in the summary dialog, not in the editing dialog.
(It is, apparently available through the google calendar web interface.)

Must be fallout from bug 1607834.

It's necessary for it to be a google calendar, otherwise the issue won't show. You can accept for a local calendar, and copy that entry into a google calendar - this will also show the issue (the local calendar can show the links, the google one can't.)

I'll take a look at this. I have a hunch where the bug is.

Ben, any update? Given how common this use case must be, this is a pretty major bug.

Yes, we have 2 different ideas how to fix it and are working on it.

I've only noticed this problem when a meeting is rescheduled. I see the hyperlinks in the calendar initially. Ingesting the revision changes the time/date but all the hyperlinks are stripped out from both the Thunderbird copy and the caldav Google calendar (but the links are provided in the updated invite). (91.0.3 Windows)

I have a patch, and I'm in the process of testing it. Testing is not easy, with Google account, Teams invitation etc.

This patch should theoretically fix it, but I cannot test it, because Google requires a phone number to login. If anybody could test it, that would be appreciated.

The fix works and fixes the bug. I've tested without patch, and I can reproduce the bug here. I've tested with the patch, and I see the plaintext description and the links, and it works as expected.

Little problem: It fails when - after accepting the invitation in TB - I edit the event in the Google web UI. Google warns me before doing that, but when I do, I see HTML source code in TB. Other case: In TB, I cannot even edit it.

This is an alternative fix, which lets us always send HTML to Google calendar, matching what the Google web UI does when modifying items. It's just as broken as Google is.

This also fixes the bug.

(same approach as Fix v3)

Same as Fix v3/v4, but add a marker what the description contains, so that one can later revert it, once Google fixes up their mess.

FWIW I manually hacked the patch into my copy of v91.0.3 and it worked as intended. I deleted a previously "mangled" invite from google calendar and TB and then re-accepted the invite and the html links were preserved in both TB and Google. Many thanks Ben.

@S.Brooks: Thanks for the verification that the fix works!

@Geoff: ping

I didn't have time to check the patch yet, but it seems a bit odd. We should be setting the plaintext as it were for the description, and not do conversions which will only get us into trouble.

We should be setting the plaintext as it were for the description

I've tried to put the plaintext in the description, but as soon as the user modifies the description in the Google web UI, we will find HTML in the description field. And nothing tells us that it changed to HTML.

The problem is that Google putting HTML into the plaintext description. So, with this patch, we go along and do the same, when syncing to Google calendar.

Comment on attachment 9239261 [details] [diff] [review]
Fix, v5

It seems like a reasonable (although horrible) thing to do in the circumstances. It's not clear to me if after sending we collect the modified item from the server in all cases. That really should happen to pick up the change and keep our copy consistent with the server copy. Please check it, my r+ is conditional.

Please JSDoc this new function.

if after sending we collect the modified item from the server in all cases. ... Please check it, my r+ is conditional.

I cannot really test "all cases". Can you give me a specific situation (with steps) to reproduce that I should check?

This fix should really be tested well by end users, and also land on TB 91, before automatic updates 78->91 are enabled.

It's this, the only use of CalDavOutboxRequest that I was concerned about but I don't know how to make that particular piece of code run. Having looked a little bit closer at it, I think it's probably not a problem that should stop this code getting out to users.

OK, that's great! Thanks for the clarificaton, Geoff.

Pushed by thunderbird@calypsoblue.org:
https://hg.mozilla.org/comm-central/rev/f3534fb14348
Send HTML in the DESCRIPTION for Google. r=darktrojan

Comment on attachment 9239261 [details] [diff] [review]
Fix, v5

[Approval Request Comment]
Regression caused by (bug #): 1607834
User impact if declined: User cannot join an MS Teams meeting, because the URLs are missing from the description.
Testing completed (on c-c, etc.): Only artificial testing done so far. Still needs backing on trunk, beta and with some real users.
Risk to taking this patch (and alternatives if risky): Very risky. Worst case, we delete the descriptions of other (non-MS-Teams) calendar entries, of the entire calendar, if something goes wrong during the conversion of the description of some calendar events. Worst case, data loss. (We don't know whether that will happen, it's the potential risk of the patch.) We can only test that with the variety of real calendar entries. Should test carefully under real world usage of real users. Affects only Google calendar users.

Comment on attachment 9239261 [details] [diff] [review]
Fix, v5

[Triage Comment]
Approved for beta

TCW, Walt, Eckard, can you test this during candidate smoketest?

Specifically, it would be good to test:

• Set up a Google calendar in TB
• Create or import events / appointments with wierd descriptions: HTML, pseudo-HTML, HTML in plaintext descriptions, and similar. Real world ones and artificially created.
• Sync to Google calendar server
• Fetch events from Google calendar server (if needed, in another TB profile)
• Check that the description does not get lost and is at least semi-readable
• For any bugs you find, double-check that it's a regression caused by this change here and does not exist in TB builds without this fix.

(In reply to Wayne Mery (:wsmwk) from comment #25)

Comment on attachment 9239261 [details] [diff] [review]
Fix, v5

[Triage Comment]
Approved for beta

TCW, Walt, Eckard, can you test this during candidate smoketest?

Sorry, I don't use Google accounts or calendars.

Thunderbird 93.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/788c7664c722

(In reply to Wayne Mery (:wsmwk) from comment #25)

Comment on attachment 9239261 [details] [diff] [review]
Fix, v5

[Triage Comment]
Approved for beta

TCW, Walt, Eckard, can you test this during candidate smoketest?

Using what's in comment 26 as STR?

Yes

So I tested by by putting some random keyboard spam stuff in the Description field, broken http link, link without https://, mailto: and they all came through fine. Where it broke was an HTML template that I use for eBay.

After adding to the Description field and saving an event, it adds "style="font-family:Arial" size="4">size="4">" with a LOT of white space before and after the template and cut off the first 21 characters of the HTML template. So yeah, something is wonky here. I don't think it's Meets specific.

Hey Arthur, thanks for your tests! Regarding your ebay HTML template, a little style change would be fine for me. But just to double-check what's happening, can you share your HTML and the exact steps for reproduction, and the result, so that I can reproduce this, and see what's going on? I want to ensure that there's nothing bigger lurking there which we haven't discovered.

(In reply to Ben Bucksch (:BenB) from comment #32)

Hey Arthur, thanks for your tests! Regarding your ebay HTML template, a little style change would be fine for me. But just to double-check what's happening, can you share your HTML and the exact steps for reproduction, and the result, so that I can reproduce this, and see what's going on? I want to ensure that there's nothing bigger lurking there which we haven't discovered.

The 1st---and apparently ONLY---time I was able to cause the Description to get mangled (see image as proof, lol), this is what I did to repro using two of my GMail account that have calendars, YMMV:

1. Create a new event using Gmail Email Account 1. Title = test Teams garbled description, set the event Time + Date for the next day, 9AM or something
2. Invite Attendee. I used Gmail Email Account 2
3. On the Description is plugged in this whole thing and saved the event

dfjkhsflkjasdhjfkladj

bugzilla.mozilla.org/show_bug.cgi?id=1727061

ttps://bugzilla.mozilla.org/show_bug.cgi?id=1727061

mailto:someone@gmail.com

<center><h2><b><font color=red>*READ THE AUCTION TERMS CAREFULLY BEFORE BIDDING!*</font></b></h2></center><p>Up for auction is ... Needs a good home.<p>I've been an eBay member since 2000 and have perfect feedback. Bid with absolute confidence. If you want positive feedback, leave positive feedback for me.<p><b>AUCTION TERMS - READ CAREFULLY!</b> - Bidders with feedback of 10 or less should contact me first before bidding. If you don't, your bid WILL BE CANCELED and you will be BLOCKED.<p>Winner to contact me within 1 day (<b>24 hours</b>) of the auction's conclusion. Transactions must be complete in 3 days (<b>72 hours</b>) of the auction's conclusion. No exceptions!<p>I use the <b>cheapest</b> USPS shipping possible and not UPS or FedEX. I ship in protective cardboard, paper or bubble mailers only and this is part of the shipping and handling charge. All shipping charges shown are for the lower 48 United States. Auctions that conclude with a value of $50 or more require domestic (US) delivery confirmation and this cost will be added to the shipping charge when I send the invoice to you. Do not pay for the auction if the value of the item has exceeded $50. Shipping will be calculated after the auction ends and is based on location. I try to ship 24-48 hours after a payment has cleared or been received (<b>Excludes: Sundays, Holidays and hours / days the Post Office is closed</b>). If I can't ship within 24-48 hours, I will do my best to notify you by e-mail.<p>UPS or FedEX tracking is <b>your</b> option if you'd like it but will cost you more and is not part of the auction's listed shipping price. Contact me after the auction concludes for alternate shipping services and the additional charges that do come with them. I retain receipts as proof of mailing. I DO NOT SHIP TO APO, FPO, or to General Delivery addresses! Once I ship an item, the length of time it takes to arrive depends wholly on the shipping service. I take no responsibility for delays caused by the shipping service. Do not bid if you do not plan to use this item. Contact me with all questions <b>before bidding</b>. Good luck!

Used the Description information from comment 33 and did not see any issue using 93.0b4 on Fedora 34 Workstation or Windows 10.

Teams invites on a google calendar are still not working properly. Likely the whole fixGoogleDescription is just plain wrong as an approach.

I got a Teams invite and the google calendar added the event (server side). The links in the event details are not shown in Thunderbird.

Teams invites on a google calendar are still not working properly.

Could you please give exact steps for reproduction?

FYI, the bug I fixed here is when you accept the MS Teams invitation in Thunderbird and then send it to a Google calendar. I think you're talking about a different case. I think you're saying that you accept the invitation within Google Calendar, right? How does the invitation get in there? Using Gmail or something else? Could you please give the concrete steps? What does Google calendar add as event description in this case, and what does Google send Thunderbird in CalDAV in this case? Could you please also post the CalDAV source document that's coming down from the Google server to Thunderbird?

I would like to plug the v5 patch into my system for testing. Can someone shed some light/point me in the direction of some instructions on where to put the patch code? If it's something I have to re-compile or super complex, I'll probably just wait until it's pushed out in a beta (unless it is already?). Wasn't sure if it was just something I could modify with a text editor on my current thunderbird install on my machine?

(In reply to chad from comment #38)

I would like to plug the v5 patch into my system for testing. Can someone shed some light/point me in the direction of some instructions on where to put the patch code? If it's something I have to re-compile or super complex, I'll probably just wait until it's pushed out in a beta (unless it is already?). Wasn't sure if it was just something I could modify with a text editor on my current thunderbird install on my machine?

The fix is in Thunderbird 93.0beta4 and beta5 (latest).

Patch working as intended when hacked into the current ESR v91.1.2
I get multiple Teams Invites a week coming into Thunderbird being saved to Google Calendar (done this way to sync channel invites to calendar on android phone). I only use the Google Calendar interface when Thunderbirds jagged edges force me to.

I struggled without clickable links in Thunderbird ESR v78 waiting more than 12 months through a pandemic in an Microsoft dominated industry. I don't want to risk my company emails to beta software and I waiting another 12 months for an ESR cycle for a fix would be soul destroying so I hope this patch can be accepted as a stop-gap into the ESR stream until such time that internet standards catch up with HTML in calendar event descriptions.

(In reply to S.Brooks from comment #40)

Patch working as intended when hacked into the current ESR v91.1.2
I get multiple Teams Invites a week coming into Thunderbird being saved to Google Calendar (done this way to sync channel invites to calendar on android phone). I only use the Google Calendar interface when Thunderbirds jagged edges force me to.

I struggled without clickable links in Thunderbird ESR v78 waiting more than 12 months through a pandemic in an Microsoft dominated industry. I don't want to risk my company emails to beta software and I waiting another 12 months for an ESR cycle for a fix would be soul destroying so I hope this patch can be accepted as a stop-gap into the ESR stream until such time that internet standards catch up with HTML in calendar event descriptions.

Hi S.Brooks! Would it be possible for you to share how you patched it in to the ESR vs me downloading the Beta? Just trying to figure out what the best route to go is.

Hacking the patch into ESR under Windows 10 (backups and common sense assumed). Hope this doesn't violate guidelines but its the only way I can keep using Thunderbird in a work setting with this bug present until a fix is mainstream.

1. close Thunderbird
2. rename "C:\Program Files\Mozilla Thunderbird\omni.ja" to omni.ja.zip
3. from the zip extract /modules/caldav/CalDavRequest.jsm
4. apply the patch ( 3 edits in notepad or use patch software) -if this is a mystery don't start.
5. reinsert the patched file into the zip
6. rename the zip back to omni.ja
7. restart Thunderbird
8. repeat whenever ESR updates until not required (hint. save the modified CalDavRequest.jsm and check modified dates in the zip)

(In reply to S.Brooks from comment #42)

Hacking the patch into ESR under Windows 10 (backups and common sense assumed). Hope this doesn't violate guidelines but its the only way I can keep using Thunderbird in a work setting with this bug present until a fix is mainstream.

1. close Thunderbird
2. rename "C:\Program Files\Mozilla Thunderbird\omni.ja" to omni.ja.zip
3. from the zip extract /modules/caldav/CalDavRequest.jsm
4. apply the patch ( 3 edits in notepad or use patch software) -if this is a mystery don't start.
5. reinsert the patched file into the zip
6. rename the zip back to omni.ja
7. restart Thunderbird
8. repeat whenever ESR updates until not required (hint. save the modified CalDavRequest.jsm and check modified dates in the zip)

So far this looks liked patching it manually worked (sending myself a quick MS meeting to thunderbird). Thanks S. Brooks and Rob!

Should we take this to 91 yet?

I applied the patch to 91.2.0.
It does not work if I view the message body as message "simple html" or "original html". It only displays the date/time of the meeting if I view the message body as "plain text".

Should I open a new bug for this?

chad, s.brooks,
Thanks for testing. Is the patch still working for you without problems?
And do you see the issue reported by michael in comment 46?

(In reply to michael.sez from comment #46)

I applied the patch to 91.2.0.
It does not work if I view the message body as message "simple html" or "original html". It only displays the date/time of the meeting if I view the message body as "plain text".

Is that still the only problem you have encountered?

Should I open a new bug for this?

Ben?

(In reply to Geoff Lankow (:darktrojan) from comment #44)

Should we take this to 91 yet?

Ben writes "it was the best fix we could find. But this is an inherently dangerous change, because we change the format of the description in the user's calendar. The fact that we didn't get a lot of complaints in beta is good, but I cannot take the responsibility for release."

Geoff, What do you and Magnus think? Pending responses to questions above, if we feel more feedback is needed, perhaps an esr try build?

I looked for other bugs reported in https://mzl.la/3g5H5kH and there is Bug 1747233 - Teams invitations not properly added to calendar

(In reply to michael.sez from comment #46)

I applied the patch to 91.2.0.
It does not work if I view the message body as message "simple html" or "original html". It only displays the date/time of the meeting if I view the message body as "plain text".

This does not make sense to me. What is the "message" that michael is viewing?

My understanding is the bug and its fix does not alter the email that arrived with the calendar invite. It changes the way the calendar invite is stored in the Google Calendar. So whether the email is viewed as plain or html should be irrelevant.

I'm surprised there are so few reports on this issue. I have been manually installing the modified CalDavRequest.jsm with the bug fix into each new release (currently using 91.5.1) to ensure my calendar behaves properly because this is my work email/calendar and I'd rather not use the beta for work emails.. yet need the calendaring to work so there is no real option but to inject the bugfix each time if I want to stay with Thunderbird.

@S.Brooks, so, you're saying that things work for you only with the fix here, so the fix substantially improves the situation for you?

Wayne already quoted my email, I am just re-posting to authenticate it:

Should we take this to 91 yet?

It was the best fix we could find. But this is an inherently dangerous change, because we change the format of the description in the user's calendar. The fact that we didn't get a lot of complaints in beta is good, but I cannot take the responsibility for release.

(In reply to Ben Bucksch (:BenB) from comment #50)

@S.Brooks, so, you're saying that things work for you only with the fix here, so the fix substantially improves the situation for you?

I have not run TB without the patch for quite some time so I'm now uncertain of how frequently I would encounter the mangled events that are missing the hyperlinks to join the meeting. At the time it was 1-2 a week (out of 4-5 invites) and was very annoying so I was pleased to see the patch. Since using it every MS Teams invite has been represented in Google Calendar properly (and reflected back to TB properly).

I will remove the patch and see if/when it happens again - just in case MS Teams has changed the formatting of the invitation emails in the last 5 months and made the problem go away - like I said before I'm surprised more people haven't encountered the bug.

I will remove the patch and see if/when it happens again

Thanks, but that's not necessary. I just wanted to clarify your earlier comment. I think it's clear now.

Since using [the patch here,] every MS Teams invite has been represented ... properly

Thank you for that experience. That gives me more confidence in the patch. I wasn't sure about negative side effects, but if you say you're running it in production since a while, and you're happy with it, that is good news.

Confirmed ESR requires the patch - after removing the patch from ESR 91.5.1 on Monday I today got a calendar invite today that was mangled as per the bug report - Teams links etc are all present in Google Calendar (so the invitation was read in by TB and stored OK in google Calendar) but when the event is viewed in Thunderbird ESR hyperlinks are removed and there is no way to join the meeting.

I reinstalled the patched CalDavRequest.jsm, deleted the event from TB and then re-accepted the invitation and the event description was restored with working MS Teams hyperlinks to join the meeting.

I will need to manually graft in the patch with every release until no longer required.

I can confirm that in TB91 on both latest Ubuntu and Win11 none of the MS Teams links contain links in google calendar. Bug 1747233 seems to track the same problem. Maybe someone can mark that one as a dupe (if it is one?) Will this fix be backported to the next tb91 release?

My report from that bug:

I am seeing this issue in tb91 with ALL MS Teams invitations (win11 and latest ubuntu).

• in the email: in the collapsed part:
  Microsoft Teams meeting
  Join on your computer or mobile app
  Click here to join the meetinghttps://teams.microsoft.com/l/meetup-join/19%3ameeting_nnnnnnnnn

• in the TB non collapsed part:
  Click here to join the meeting (which is a hyperlink with the above link behind it)

• in the TB google calendar stored invitation (possibly related bug 1659363 / bug 1727061 ??) in the description window:
  Click here to join the meeting (without a link behind it)

• in the iOS stored calendar the link is shown.

• on iOS the google calendar the link is gone as in the TB view

Please let me know if if is this issue, bug 1659363 / bug 1727061 , or if I should file a new issue for it. If you need more info, happy to provide it...

Thanks!!

My workaround: make sure I don't delete the mails... accept them, move them to a subfolder (otherwise if I accept them in iOS in parallel the email is deleted), and then hope I find the email to click on the meeting...

Using ThB "91.5.0 (64-bit)".

I encounter multiple problems with Teams invitations:

• When I initially received the Email, there is absolutely no information on when (date / time) it is supposed to occur. You have to accept the meeting even before knowing if you will be free at this time (& once accepted, you still have to look for the event in the calendar...) !
• [Issue stated here] When I accept the item (and probably before that too), I have no link/info about how to join the event (see attached screenshot: text before highlighted "pour" word is not clickable).

I don't know if it is Teams/MS and/or GCal/Google's fault, but it make Thb/Mozilla difficult to operate with collaborators that use only proprietary software :{

Link to screenshot
https://ibb.co/D1Dxh2n

(In reply to L.I.A.R. from comment #56)

Link to screenshot
https://ibb.co/D1Dxh2n

When I click the link in FF or Chrome it goes to a blank white screen or gives an error. Maybe it pasted wrong?

Everybody: Could you please try TB Beta (not TB 91) and see whether the problem is fixed for you, and how well it's working for you? The fix for this has been in beta since 5 months, and we're basically waiting for feedback from users to know how well is works. You find the beta at https://www.thunderbird.net/ on the bottom right. Thank you.

What we'd like to know is: Does TB beta work for you, with a) new Teams events which you added using TB beta? b) existing Teams events which you added with TB 91? c) Does TB beta break any calendar entries in new (!) ways, something that worked fine in TB 91 and now breaks (a regression), or does everything else still work as TB 91 did? Particularly the last question is important for us.

I just downloaded latest available beta (98.0b2) and so far I can comment only on

b) existing Teams events which you added with TB 91?

old events are still not rendering properly - the URL to join the meeting is still missing in the description (when I view this calendar entry via Google calendar, it renders fine)

Also TB98b2, Teams invitations in google calendar still do not show links. I will test with new invites as soon as I get them in the next few days. No regressions that I can see so far in calendaring.

Yes, that's not surprising that old items don't get fixed retrospectively. Please let us know about questions a) and c) in comment 59. Thanks!

I may have a regression, or it may be a separate bug. I can't (with TB98b2) accept changes to individual events (i.e. moved from 10:00 to 11:00) and I can't accept them for series events (one of a series or the whole series changed) either.

It used to at least accept the change (though it sometimes actually changed the calendar entry and sometimes not, I never looked into it in detail), but now clicking on "Accept" does nothing. Something that might be related: I used to have to accept a meeting, then select the calendar, and then accept the meeting again for the status to change to "...already been processed" Please let me know if you think this may be connected, point me to another bug, or let me know if I should file a new one.

@Lohner: This seems unrelated to the event description and unrelated to this change here. Can you file a new bug, please?

Done in Bug 1756547

Tested in TB98b3... does not work. In the email the description is now unfolded (that was another bug, now the date / time of the meeting is visible without having to click, yay!) but accepting the invitation seems to be different than before:

• in TB itself when going to Details / clicking on the calendar event (or hovering over the event): there are NO links:

Microsoft Teams meeting
Join on your computer or mobile app
Click here to join the meeting
Learn More | Meeting options

• opening the accepted event in iOS DOES show the link, so apparently links are now getting stored in the google calendar but links are not shown in TB

Hi Nils, it's difficult to tell which view you're looking at, whether it's the preview or the event editor. Could you please file a new bug, with all steps? It's important to list every click, every step, and all data sources. Please concentrate on one bug only - mentioning other issues at the same time will only confuse us and be detrimental to fixing it. Include screenshots wherever possible.

I see it as plain text without any links when I a) hover over the event in the calendar as text only, b) when I double click the event in the calendar under the "description" field and c) same when I click on the "details" in the email invitation itself.

The Teams event was sent to me via Outlook, and I accepted it into my google calendar. The only reason I mentioned that the link showed up on iOS is that this means that the storage of the event with links seems to work now, just the displaying with links does not.

If I understand you correctly you would like me to file a new bug for the links not being displayed, now that they appear to be stored? Done in Bug 1758438