User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36 Edg/85.0.564.41

Steps to reproduce:

In Thunderbird 68 and earlier we use MCD autoconfig to read user information for LDAP
..Thunderbird\Default\pref\autoconfig.js points to autoconfiguration script that uses
function processLDAPValues (values) to read users attributes.
Starting Thunderbird 69 the MCD script does not return values.
Is any way to fix MCD or any other way to auto setup users profiles and preferences?

Actual results:

MCD script and ProcessLDAPValues function does not read user information. Sets LDAP address books, and other preferences.

Expected results:

MCD autoconfig script reads user information from LDAP and sets user account and profile preferences.

Does anybody has any latest information how to make MCD autoconfig work or any other information how to setup user profile/account in Enterprise environment.
Any sample scripts that work with Thunderbird 78.

Thank you

https://mail.mozilla.org/pipermail/tb-enterprise/2019-October/001604.html ?

In TB 68 we did disable Sandbox to make autoconfig script work.

This is C:\Program Files (x86)\Mozilla Thunderbird\defaults\pref\autoconfig.js
//Pointer to Autoconfiguration script
pref('general.config.obscure_value',0);
pref('general.config.filename', 'Mozilla.cfg');
pref('general.config.vendor', 'Mozilla');
pref('general.config.sandbox_enabled', false);

Any other new features in TB 69 and later, block MCD autoconfig scripts?

Hmmm, the document on MDN presents MCD (Mission Control Desktop) in "Archive of obsolete content"... ?

https://developer.mozilla.org/en-US/docs/Archive/Misc_top_level/MCD,_Mission_Control_Desktop_AKA_AutoConfig

(more via matrix-PM to Wayne today)

I'm very confused. I have people that said it didn't work in 68. You're saying it broke between 68 and 78?

Moving to LDAP. Not an Autoconfig issue.

If LDAP is working correctly, then this isn't an LDAP issue.
It was necessary to change LDAP, as described in the other bug, because of platform changes.

If that change caused something else to fail - then that other code needs to be adjusted, IIUC.

I don't know the autoconfig code, so I'm purely speculating. Could the autoconfig code currently assume that a call will synchronously return data, which it doesn't?

Then you need to track down which code that is, and change it to wait for results asynchronously - potentially this requires introducing some new mechanism that calls a callback - once data is available.

The code is here:

https://searchfox.org/mozilla-central/source/extensions/pref/autoconfig/src/prefcalls.js#157

It's calling

https://searchfox.org/comm-central/source/ldap/xpcom/public/nsILDAPSyncQuery.idl#24

which is a sync function by design.

This should probably be fixed as a matter of priority, as it prevents large organisational/enterprise consumers with thousands of users from deploying/using Thunderbird.

• Magnus, should this be assigned to someone?
• Ben and Kai: Per Mike's analysis (summarized in Bug 1592922 Comment 37), after he has fixed the MCD API part in bug 1592922, this is now a regression caused by our bug 1576364, which is said to break LDAP in autoconfig. Do you agree? Is the failure understood, maybe along Kai's comment 7 combined with Mike's reply of comment 8, sync vs. async? Any ideas how to address this?

With the latest information, I agree, it looks like a regression in LDAP code.

I don't know how to fix this at this time, this needs analysis.

I agree that fixing MCD should be a priority.
Users love Thunderbird as email client, but if we can not autoconfigure user profiles, we will be forced to look for alternatives.

Seems like this should P1, so that we don't have users languishing on version 68

Mike, thanks for the help so far!

Is it possible to mark getLDAPAttributes async in the API? https://searchfox.org/mozilla-central/rev/c409dd9235c133ab41eba635f906aa16e050c197/extensions/pref/autoconfig/src/prefcalls.js#228 (and do other needed changes to make it return the values, when it has them)

I don't quite understand how the API is used? Are these functions that can be put into js code for autoconfig.js?

I tied calling getLDAPAttributes async, but it didn't work.

the issue is the entire CPP file:

https://searchfox.org/comm-central/source/ldap/xpcom/src/nsLDAPSyncQuery.cpp

Is built around Sync LDAP.

This patch should fix it.
There are a couple of things happening here:

1. I broke the nsLDAPSyncQuery when I added the onLDAPError() callback to nsILDAPMessageListener. (The "part-three" patch applied for Bug 1659947). This means it would hang if an LDAP error occured.
2. nsLDAPSyncQuery.getQueryResults() didn't really report any errors. This patch makes sure that meaningful errors are thrown.

CAVEAT:

• With this patch, I can connect and query my local slapd server, but my server isn't set up with any data, so I haven't tested a query that actually returns anything! I'm using a unit test file to drive it, and unit tests aren't allowed to connect to non-local servers. So it'd be good to see this working in a real use case. Any volunteers?

NOTES:

• previously, connection errors (eg no server at the given host/port) wouldn't cause an error, just an empty return string. Now connection errors will be thrown up. This might potentially trip up existing uses? (but it's still the Right Thing(tm) to do)
• security errors can now be caught here (eg connecting to a server with a self-signed cert, but without having that certificate registered as a security exception in Thunderbird). So you can check for this by looking at the error code. However, there's currently no way to access the failing certificate (so you can't pop up the add-an-exception dialog and retry). I'm not sure this would be desirable for enterprise-style setups or not... If needed I can add a mechanism to access the failed certificate when such an error occurs (probably by adding a .failedSecInfo attribute to the nsILDAPSyncQuery interface).

Just in case it's useful to anyone else - here's my hacky little unit test file for driving the nsLDAPSyncQuery.
Would be nice to turn it into a proper set of tests and add it, but we don't currently have a dummy LDAP server to run against.

I'll take care of building and testing. It looks like this doesn't make ldapQuery async so no change needed to the caller?

(In reply to Mike Kaply [:mkaply] from comment #18)

I'll take care of building and testing. It looks like this doesn't make ldapQuery async so no change needed to the caller?

Yep, it's just a fix to the existing implementation. Same API.

(In reply to Ben Campbell from comment #17)

Would be nice to turn it into a proper set of tests and add it, but we don't currently have a dummy LDAP server to run against.

We do actually have an LDAP mock server now, since bug 1608304. Check browser_ldap_search.js and others. Doesn't do LDAP TLS (yet) though.

I gave this a try and it works differently (blank instead of null), but still doesn't show results.

I've attached a firefox.cfg that works in Thunderbird 68.2.2. It shows two results on the JS console at startup.

You can use this autoconfig.js in defaults/pref to enable it.

pref("general.config.obscure_value", 0);
pref("general.config.filename", "firefox.cfg");
pref("general.config.sandbox_enabled", false);

This is what I see in the console on 62.2.2

givenName=Aaron
givenName=Sheryl firefox.cfg:11
processLDAPValues firefox.cfg:11
getLDAPAttributes prefcalls.js:156
<anonymous> firefox.cfg:16

Note I didn't add a try/catch to check for errors. I'll do that next.

No errors, and it's definitely blank.

OK, obviously something else going on as well. I'll keep investigating. Having the script in Comment 21 and a mock LDAP server should help a lot.
I think the patch is still valid - all the cool kids are handling errors these days - but yeah, not much use if it doesn't actually return any data!

That cfg file I attached should make it really easy to test.

Thanks Mike - that was very helpful!
I think I've got it working. Seems like a simple fix, and I'm not entirely sure how it ever worked in the past...

Anyway. Want to give this one a try and see if it does what you want?
The previous error-handling patch is still required. I was going to roll it all into one, but it seems there are two separate and unrelated issues to fix, so I'll leave them separate.

If this does fix things, I'll code up some unit tests to go with it using our shiny new fake LDAP server.

(In reply to Mike Kaply [:mkaply] from comment #22)

Note I didn't add a try/catch to check for errors. I'll do that next.

Ahh yes. That won't work unless you write your own getLDAPAttributes() function. The one there in M-C catches the exception and displays an error dialog before it returns.

Actually given that prefcalls.js is the only place in M-C that seems to reference any of the nsILDAP classes, there's probably an argument for removing all the LDAP helpers there and make people supply their own LDAP code... (maybe with some helpers in C-C to make it trivial).

Actually given that prefcalls.js is the only place in M-C that seems to reference any of the nsILDAP classes, there's probably an argument for removing all the LDAP helpers there and make people supply their own LDAP code... (maybe with some helpers in C-C to make it trivial).

I thought about that, but I'm not sure it's worth the effort, especially if it means folks having to modify their Autoconfig. This is stuff that has worked for 10+ years, so probably not worth reinventing.

As far as the try/catch goers, the actual call to query the server is in prefcalls.js, so we should be able to try/catch it there for failure, shouldn't we?

(In reply to Mike Kaply [:mkaply] from comment #28)

Actually given that prefcalls.js is the only place in M-C that seems to reference any of the nsILDAP classes, there's probably an argument for removing all the LDAP helpers there and make people supply their own LDAP code... (maybe with some helpers in C-C to make it trivial).

I thought about that, but I'm not sure it's worth the effort, especially if it means folks having to modify their Autoconfig. This is stuff that has worked for 10+ years, so probably not worth reinventing.

Yup - that's kind of what I figured. It'll all be in-house too, so no real public record of anyone using this stuff at all, even if it's critical for their organisation!

As far as the try/catch goers, the actual call to query the server is in prefcalls.js, so we should be able to try/catch it there for failure, shouldn't we?

That's what I meant - prefcalls.js is already catching the error and displaying a dialog box, so if you wanted to catch and handle any error conditions yourself (eg installing certificate exceptions, say), you'd have to do the server call yourself.

Comment on attachment 9184417 [details] [diff] [review]
1662433-add-error-handling-to-nsLDAPSyncQuery-1.patch

Shouldn't this one be returning rv of the converted status?

@@ -154,7 +159,7 @@ nsresult nsLDAPSyncQuery::OnLDAPSearchEn
NS_WARNING(
"nsLDAPSyncQuery:OnLDAPSearchEntry(): "
"aMessage->GetAttributes() failed");

• FinishLDAPQuery();

• FinishLDAPQuery(NS_ERROR_FAILURE);
  return rv;
  }

Patch works for me. Thank you!

I'm getting the Autoconfig fix into mozilla central.

Also, I caused some things to error out, and it worked as expected, showing the errors. So I don't think we need any changes to prefcalls.

Minor tweak to the error-handling patch, based on Comment 30 (Also I realised I'd forgotten to set my diff output context to 8 lines instead of the default 4!)

Mike - you've obviously had a peek through the code, so I shall lumber you with a review for my two patches :-)

Any progress with getting the tests for it going?

I did start looking at this, but the existing fake LDAP server requires the test to manual pump data in and out of it, which means it's tricky to use with the nsLDAPSyncQuery (which doesn't return until the query completes). I think it'd be possible to use some javascript voodoo to get around this, but I'm rather leaning toward implementing more of the LDAP server - just enough to load it up with some test data and handle enough of the protocol to cover our pretty modest requirements.
Will investigate, and maybe break it out into a new bug.

These patches need un-bitrotting before landing

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/2538e2f3517e
Add error handling to nsLDAPSyncQuery.getQueryResults(). r=mkaply
https://hg.mozilla.org/comm-central/rev/c8169ad6ff96
Fix result attribute iteration & output in nsLDAPSyncQuery. r=mkaply

Comment on attachment 9185300 [details] [diff] [review]
1662433-fix-nsLDAPSyncQuery-output-2.patch

[Approval Request Comment]
User impact if declined: MDC ldap setup doesn't work
Testing completed (on c-c, etc.): on c-c, beta
Risk to taking this patch (and alternatives if risky): it's broken as is

Approval for both patches.

Comment on attachment 9185300 [details] [diff] [review]
1662433-fix-nsLDAPSyncQuery-output-2.patch

[Triage Comment]
Approved for esr78

Comment on attachment 9185298 [details] [diff] [review]
1662433-add-error-handling-to-nsLDAPSyncQuery-3.patch

[Triage Comment]
Approved for esr78

Thunderbird 78.5.1:
https://hg.mozilla.org/releases/comm-esr78/rev/6c60a6a605a8
https://hg.mozilla.org/releases/comm-esr78/rev/ebb170e1fa47

Hello
I had tried to plug the same autoconfig settings as in 68 to Thunderbird 78.5.1, but it seems it is not even reading config file.
Anybody had any success in making MCD working in Thunderbird?

Configuration below worked for all Thunderbird versions up to 68.3
...default\prefs\autoconfig.js

pref('general.config.obscure_value',0);
pref('general.config.filename', 'mozilla.cfg');
pref('general.config.vendor', 'mozilla');
pref('general.config.sandbox_enabled', false);

mozilla.cfg
lockPref("general.config.vendor", "oakton");
lockPref("autoadmin.global_config_url","https://******/thunderbird.cgi");
lockPref("autoadmin.append_emailaddr",false);
thunderbird.cgi reads .cfg that runs processLDAPValues function.
But it is not even reading configuraiton file, usually at this point failover.js is created in user profile.

Please advise.
Thank you

Hi Mike, any idea what might be wrong with the autoconfig setup of this user?
I've tried to offer some informed guesses...

(In reply to marinar from comment #43)

Hello
I had tried to plug the same autoconfig settings as in 68 to Thunderbird 78.5.1, but it seems it is not even reading config file.
Anybody had any success in making MCD working in Thunderbird?

Configuration below worked for all Thunderbird versions up to 68.3
...default\prefs\autoconfig.js

pref('general.config.obscure_value',0);
pref('general.config.filename', 'mozilla.cfg');

Hi Marinar, can you please try to move the following two lines into mozilla.cfg, and report back?
All the examples are just showing two lines of code in autoconfig.js as a pointer to actual configuration file.
Maybe it won't read more than the top two lines, which would make sense.

pref('general.config.vendor', 'mozilla');
pref('general.config.sandbox_enabled', false);

mozilla.cfg
lockPref("general.config.vendor", "oakton");
lockPref("autoadmin.global_config_url","https://******/thunderbird.cgi");

Also, I am not sure if the .cgi in between will work, because you're asking Thunderbird to run this file and I'd think it expects javascript files only, as the example in documentation suggests:

Centralized Management
The AutoConfig file can be managed centrally. To do so, reference the location of
a secondary AutoConfig file in the primary AutoConfig file:
pref("autoadmin.global_config_url","https://yourdomain.com/autoconfigfile.js");

lockPref("autoadmin.append_emailaddr",false);
thunderbird.cgi reads .cfg that runs processLDAPValues function.
But it is not even reading configuraiton file, usually at this point failover.js is created in user profile.

Please advise.
Thank you

Hello
Thank you for suggestions.
I tried these changes, but configuration did not work.

The described above setup was working for 15 years and originally were setup from this article
https://developer.mozilla.org/en-US/docs/Archive/Misc_top_level/MCD,_Mission_Control_Desktop_AKA_AutoConfig

Currently above setup works for Thunderbird 68.2.2 with pref('general.config.sandbox_enabled', false);

Any documentation how to setu Mission Control Autoconfig for Thunderbird 78

Thank you

I verified that local autoconfig works fine with getLDAPAttributes in the local CFG file.

We didn't test getLDAPAttribute in a remote CFG file.

My autoconfig.js is:

pref("general.config.obscure_value", 0);
pref("general.config.filename", "firefox.cfg");
pref("general.config.sandbox_enabled", false);

My mozilla.cfg is:

//
var ldapHost = "ldap.adams.edu";
var ldapBase = "ou=people,dc=adams,dc=edu";

var ldapFilter = "surname=Abeyta";

var ldapAttrs = new Array( "givenname" );

// Define how to process LDAP results before we make the call
function processLDAPValues(queryResults)
{
Components.utils.reportError(queryResults);
}

// Call upon LDAP for the values in ldapAttrs array
// Uses the previous processLDAPValues()
getLDAPAttributes( ldapHost, ldapBase, ldapFilter, ldapAttrs.join(",") );

I'll try to convert this into a remote file for testing.

I converted to a remote file and it worked fine for me.

Can you please run thunderbird with --jsconsole and see if there are any errors?

Thank you very much.

Autoconfiguration is working now from local cfg and directly from remote cfg, bypassing cgi call script in the middle.

Really appreciate your help.

Note: Some older code should be removed from .cfg, otherwise Thunderbird can not read configuration file.
Components.utils.import("resource:///modules/gloda/log4moz.js");
var log = Log4Moz.getConfiguredLogger("MCD");

Magnus, apart from the file name change (.js --> .jsm), is there anything else which could prevent enterprise users from importing this resource via MCD AutoConfig? Does the long form Components.utils work here or they must use Cu instead?

Thunderbird.cfg (AutoConfig file, local or external)

Components.utils.import("resource:///modules/gloda/log4moz.jsm");
var log = Log4Moz.getConfiguredLogger("MCD");

(In reply to marinar from comment #48)

Thank you very much.
Autoconfiguration is working now from local cfg and directly from remote cfg, bypassing cgi call script in the middle.

Really appreciate your help.

Most welcome, we're here to support you so that your enterprise scenarios will work well. So the .cgi was indeed part of the problem as I hinted in my comment 44.

Note: Some older code should be removed from .cfg, otherwise Thunderbird can not read configuration file.
Components.utils.import("resource:///modules/gloda/log4moz.js");

Marinar, this module still exists, but the file name has changed, it's now log4moz.jsm.
Can you please try again with the correct file name and report back?

https://searchfox.org/comm-central/source/mailnews/db/gloda/modules/Log4moz.jsm

 var log = Log4Moz.getConfiguredLogger("MCD");

This should still work after the file name correction to .jsm, unless if we have another permissions/API problem here. Note that the logger name is case sensitive.

https://searchfox.org/comm-central/rev/5640a40e0d1d800e461050adadc46e29bdbe3982/mailnews/db/gloda/modules/Log4moz.jsm#47-72

(In reply to Thomas D. (:thomas8) from comment #44)

lockPref("autoadmin.global_config_url","https://******/thunderbird.cgi");

Also, I am not sure if the .cgi in between will work, because you're asking Thunderbird to run this file and I'd think it expects javascript files only, as the example in documentation suggests

Thanks a lot Mike for your expertise and guidance here!

(In reply to Mike Kaply [:mkaply] from comment #46)

I verified that local autoconfig works fine with getLDAPAttributes in the local CFG file.

(In reply to Mike Kaply [:mkaply] from comment #47)

I converted to a remote file and it worked fine for me.

(In reply to Thomas D. (:thomas8) from comment #49)
Any code constructs that have changed/been removed obviously won't work. Cu will work, but Cu.import doesn't import the same way as before. You can find code refs on https://searchfox.org/
Don't add log4moz usage either since we'll get rid of that soon.

If anyone has problems, strip down the file to minimum, without additional imports or other custom code that may be obsolete.

Hello

Just for information since it will be removed soon.
I tested configuration file with
Components.utils.import("resource:///modules/gloda/log4moz.jsm");
var log = Log4Moz.getConfiguredLogger("MCD");
and received error "Failed to read the configuration file. Please contact your system administrator."

Works perfectly without those lines

Thank you