Implement cross-platform SPNEGO. This bug is about providing a cross-platform SPNEGO implementation in Mozilla similar to the one included with Samba's libsmb. Using OS provided SPNEGO implementations (via gssapi or sspi) is suboptimal for a number of reasons: 1) under GSSAPI there is no way to negotiate NTLM since GSSAPI does not implement NTLM. 2) under SSPI, NTLM may be negotiated. the result is that SSPI may use its internal implementation of NTLM, which is undesirable. SPNEGO described by RFC 2478 requires an ASN1 encoder/decoder, which NSS provides. The goal of this bug is therefore to write a nsIAuthModule implementation that will live in PSM (along side nsNTLMAuthModule). We'll also want to move extensions/negotiateauth to extensions/krb5auth since we'll still need to support KRB5 as an optional feature dependent on system libraries.

The mozilla SPNEGO should not be used if the system already has its own SPNEGO support. Example - Solaris Express (and Solaris 10) has a fully functional SPNEGO module that works with the native GSSAPI library. The configuration should check for this when building the code in Solaris. MIT Kerberos may have their own SPNEGO implementation in the future as well. Ditto Heimdal. GSSAPI does not implement any specific mechanisms, its just a generic layer. Someone could easily write an NTLM plugin for GSSAPI to use just like there are KRB5, Diffie-Hellman, SPKM, and other mechanisms. SPNEGO should negotiate whatever is available on the system. Because MS chooses to use NTLM and Kerberos today does not mean that they will always offer just those 2 choices, they may well choose to implement new auth methods (SPKM, PKINIT, etc) in the future. Its incorrect to override an existing system GSSAPI/SPNEGO implementation with a local copy unless you know for sure that the system's implementation is fatally flawed in some way and the vendor is not going to fix it. I hate to rant here, but one of the more valid criticisms of Mozilla is the bloating and feature creep. IMO, this is a case where mozilla should not try to re-invent or include a home-brew version of code that is (or will be) available elsewhere.

wyllys: i appreciate your feedback, but how do you propose that we solve the two problems i outlined? SPNEGO is a lightweight protocol. just look at the implementation in libsmb... we're talking about less than 350 lines of code. we just need to decode the ASN1 packets, extract the mechs, and then talk to the OS layer to make use of available auth schemes. the advantage to having mozilla do its own SPNEGO decoding is that mozilla can then ensure that the right things happen consistently across all platforms. does samba disable its version of spnego when an implementation is available on the host system? i don't think it does. i'm not trying to limit mozilla to only krb5 and ntlm. instead, i'm trying to make sure that it uses ntlm when ntlm is not provided by the operating system, which is the case under Linux and OSX. i don't really have that strong of an opinion on what is compiled into mozilla for the solaris version, but it seems to me that it would be best to have a consistent behavior across all platforms.

If SPNEGO in libsmb is only 350 lines then its most likely an imcomplete implementation of the spec. They probably implemented just exactly the bits that they needed and ignored the rest. The ASN.1 needed for SPNEGO is pretty trivial, btw, so leveraging the ASN.1 in NSS may or may not be necessary. If you look at the code that Microsoft gives away on their website for implementing SPNEGO with HTTP NegotiateAuth, it is also very incomplete and only implements exactly what is needed (it's also completely broken with respect to the IETF specification, but thats another story...) for that exchange. I'm wondering just how useful and desirable is NTLM authentication for Unix users as a whole. Most sites (yes, I'm probably generalizing here) that implement Active Directory are using Kerberos as the primary authentication name service and not NTLM (if they have a shred of decency and concern for security). I'd be tempted to tell anyone who really wants NTLM in Unix to write it themselves and don't pollute mozilla with that ugliness. Anyway, SPNEGO was designed to be implemented as a pseudo-mechanism for GSSAPI, not as an independent library, separated from GSSAPI. If you add your own SPNEGO to mozilla, how is it going to handle other mechanisms that might be negotiated in the future? GSSAPI is designed to be pluggable and extensible without requiring a recompile of applications that use it. MIT will be moving towards a pluggable multi-mech GSSAPI implementation in the future (there are already talks underway to make this happen sooner rather than later). Keeping Mozilla talking only to the GSSAPI layer ensures that it will be able to support other mechs in the future without having to be recompiled and redistributed. I think SAMBA is bad example to follow for coding practices, so the fact that they implemented their own SPNEGO instead of augmenting MIT or Heimdal was a poor decision, and one that should not be repeated, IMO. I agree that consistent behavior across platforms is a good goal, but not if that means limiting the functionality to the lowest form available because not all platforms have the necessary features. If the underlying system, (be it Linux, OSX, Win2K, Solaris, or whatever) has compatible features that are stronger and better integrated into the OS, then mozilla should attempt to use the native implementation. I put alot of work into the configuration of the mozilla code to make sure it could use Native GSSAPI, MIT, or Heimdal since all have valid implementations of Kerberos and GSSAPI, I don't want to lose that feature in the future, it will break alot of interop features and make administration more difficult. I say don't bother with this for now, wait a while until MIT has a multi-mech GSSAPI implementation, then you can just implement an NTLM GSSAPI mechanism to go underneath it and be done.

> If SPNEGO in libsmb is only 350 lines then its most likely an imcomplete > implementation of the spec. They probably implemented just exactly the > bits that they needed and ignored the rest. > > The ASN.1 needed for SPNEGO is pretty trivial, btw, so leveraging the ASN.1 in > NSS may or may not be necessary. We implemented complete support for SPNEGO. We certainly didn't implement the additional GSS-API calls discussed in the RFC since we don't export a GSS-API interface. That's only two calls though. > If you look at the code that Microsoft gives away on their website > for implementing SPNEGO with HTTP NegotiateAuth, it is also very incomplete and > only implements exactly what is needed (it's also completely broken with respect > to the IETF specification, but thats another story...) for that exchange. The story I've been told is that Microsoft contracted someone to write that example code. They wrote it to the RFC2478 specification. The RFC2478 specs are slightly different that the Microsoft implementation (the format of the NegTokenInit::mechListMIC is different, but otherwise, it's the same). > I'm wondering just how useful and desirable is NTLM authentication for Unix > users as a whole. Most sites (yes, I'm probably generalizing here) that > implement Active Directory are using Kerberos as the primary authentication name > service and not NTLM (if they have a shred of decency and concern for security). > I'd be tempted to tell anyone who really wants NTLM in Unix to write it > themselves and don't pollute mozilla with that ugliness. The whole point of SPNEGO is to be able to negotiate seperate mechanisms. HTTP-Negotiate can do raw KRB5. GSS-API offers no way for a server and client to negotiate which authentication mechanism to use. Implementing SPNEGO with only a single mechanism seems to be a waste (that's what GSS-API is for in the first place). As for the necessity of implementing NTLMSSP encapsulated in SPNEGO, there are certainly many circumstances where Windows falls back to NTLMSSP. Time skews, misconfigured DNS, and broken firewall configs can all lead to temporary fallbacks to NTLMSSP. The question is whether Mozilla should work in all the same circumstances as Windows. > Anyway, SPNEGO was designed to be implemented as a pseudo-mechanism for GSSAPI, > not as an independent library, separated from GSSAPI. SPNEGO is a GSSAPI mechanism used to negotiate an authentication mechanism. > If you add your own SPNEGO to mozilla, how is it going to handle other > mechanisms that might be negotiated in the future? GSSAPI is designed I think the argument here is that various platforms provide various degrees of support for protocols that SPNEGO would support. Having a consistent user experience on all platforms is probably desirable. > applications that use it. MIT will be moving towards a pluggable multi-mech > GSSAPI implementation in the future (there are already talks underway to > make this happen sooner rather than later). Keeping Mozilla talking There's been talk of this for years now. Like most OS projects, MIT has limited resources. > I think SAMBA is bad example to follow for coding practices, so the fact > that they implemented their own SPNEGO instead of augmenting MIT or > Heimdal was a poor decision, and one that should not be repeated, IMO. Ouch :-) I'll admit the Samba code isn't the prettiest but the advantage we do have is that we've tested again most major vendors out there. I also don't see how MIT or Heimdal is going to support something like NTLMSSP... > should attempt to use the native implementation. I put alot of work into > the configuration of the mozilla code to make sure it could use > Native GSSAPI, MIT, or Heimdal since all have valid implementations of > Kerberos and GSSAPI, I don't want to lose that feature in the future, it > will break alot of interop features and make administration more difficult. The lack of consistency (and the resulting configuration nightmare) between GSS-API implementations is worrysome. We've got a ton of code to handle these differences in Samba. Again, the Kerberos community only has limited resources to work on standardization so this is kind of something we have to live with. Of course, I'm not aware of many platforms that provide solid SPNEGO support (that includes NTLMSSP) other than Windows and even on Windows that support is not consistent. I'm sure it's possible to have configuration within Mozilla such that if a platform provides better support for SPNEGO, that is used instead. > I say don't bother with this for now, wait a while until MIT has a > multi-mech GSSAPI implementation, then you can just implement an > NTLM GSSAPI mechanism to go underneath it and be done. This could be a long time from now. This is just beginning to be discussed on krbdev. Of course, you still have to deal with platforms that provide neither Heimdal nor MIT that need SPNEGO (like Win32).

(In reply to comment #4) > > The story I've been told is that Microsoft contracted someone to write that > example code. They wrote it to the RFC2478 specification. The RFC2478 specs > are slightly different that the Microsoft implementation (the format of the > NegTokenInit::mechListMIC is different, but otherwise, it's the same). That slight difference is pretty significant if you are trying to write an interoperable implementation. It pretty much makes it impossible to be both RFC compliant and MS-interoperable. There were other issues as well, but I don't recall them at the moment. > > > I'm wondering just how useful and desirable is NTLM authentication for Unix > > users as a whole. Most sites (yes, I'm probably generalizing here) that > > implement Active Directory are using Kerberos as the primary authentication name > > service and not NTLM (if they have a shred of decency and concern for security). > > I'd be tempted to tell anyone who really wants NTLM in Unix to write it > > themselves and don't pollute mozilla with that ugliness. > > The whole point of SPNEGO is to be able to negotiate seperate mechanisms. > HTTP-Negotiate can do raw KRB5. GSS-API offers no way for a server and client > to negotiate which authentication mechanism to use. Implementing SPNEGO with > only a single mechanism seems to be a waste (that's what GSS-API is for in the > first place). I agree SPNEGO with only 1 mechanism is a waste. However, just to clarify, I don't think that the MS NegotiateAuth exchange works with "raw" KRB5, it does work with GSSAPI and KRB5 (i.e. GSSAPI tokens using a Kerberos mechanism), but does not support the exchange of raw Kerberos tickets that are not wrapped in GSSAPI tokens. > > As for the necessity of implementing NTLMSSP encapsulated in SPNEGO, there are > certainly many circumstances where Windows falls back to NTLMSSP. Time skews, > misconfigured DNS, and broken firewall configs can all lead to temporary > fallbacks to NTLMSSP. The question is whether Mozilla should work in all the > same circumstances as Windows. Perhaps on Windows it can, by just using the same SSPI interface that MS uses. The original goal of the negotiateauth mechanism support was to provide single-sign on capability. NTLM is not used for cross-platform SSO, only Kerberos is (today). So the lack of NTLM support on Unix platforms for mozilla is no big deal, IMO. > > Anyway, SPNEGO was designed to be implemented as a pseudo-mechanism for GSSAPI, > > not as an independent library, separated from GSSAPI. > > SPNEGO is a GSSAPI mechanism used to negotiate an authentication mechanism. Right. GSSAPI mechanisms are supposed to be used underneath GSSAPI itself, not directly. The whole point of GSSAPI is to provide a generic interface that apps can depend on and link against and not have to recompile or rebuild the apps whenever the underlying mechanism changes. An app that writes to the GSSAPI spec should only need to specify the SPNEGO OID in order to specify the use of SPNEGO. Likewise, that app could then just switch to specifying the KRB5 OID to use Kerberos. The app should not have to switch APIs or talk to a different library to get SPNEGO support. > > > If you add your own SPNEGO to mozilla, how is it going to handle other > > mechanisms that might be negotiated in the future? GSSAPI is designed > > I think the argument here is that various platforms provide various degrees of > support for protocols that SPNEGO would support. Having a consistent user > experience on all platforms is probably desirable. In order to use this feature, the platform must already have Kerberos and GSSAPI on the system. This means they already have all the necessary pieces, either as part of MIT, Heimdal, Solaris, or whatever the platform provides. Since MIT and Heimdal seem to be the most popular implementations, why not work with one (or both) of them to make their GSSAPI implementations more robust and support SPNEGO instead of making a private copy of SPNEGO that won't work with the existing ones on those platforms ? [...] > The lack of consistency (and the resulting configuration nightmare) between > GSS-API implementations is worrysome. We've got a ton of code to handle these > differences in Samba. Again, the Kerberos community only has limited resources > to work on standardization so this is kind of something we have to live with. > Of course, I'm not aware of many platforms that provide solid SPNEGO support > (that includes NTLMSSP) other than Windows and even on Windows that support is > not consistent. I'm sure it's possible to have configuration within Mozilla > such that if a platform provides better support for SPNEGO, that is used instead. > That's what I'm looking for - some assurance that the mozilla platform will not be limited to only its own GSSAPI and SPNEGO implementation. Solaris has a pluggable, multi-mech, GSSAPI interface and has DH, KRB5, and SPNEGO mechanism support that works very well with mozilla today. I don't want to lose that if someone decides to write their own SPNEGO for mozilla and thus ignore the platform provided GSSAPI/KRB5/SPNEGO functionality. MS has a fully functional SSPI interface that has SPNEGO (albeit broken), KRB5, NTLM, and I'm sure more in the future. The fact that Linux and *BSD are lacking this support today is indeed troubling, but I would prefer to see the effort put into making the MIT or Heimdal implementations more robust instead of adding a local implementation to be carried around with each application (mozilla, SAMBA, etc). I'm sure MIT or Heimdal would gladly consider the code for inclusion in their distributions if someone would go to the trouble of writing it and making it work correctly. It would then benefit everyone and not just a particular app. > > I say don't bother with this for now, wait a while until MIT has a > > multi-mech GSSAPI implementation, then you can just implement an > > NTLM GSSAPI mechanism to go underneath it and be done. > > This could be a long time from now. This is just beginning to be discussed on > krbdev. Of course, you still have to deal with platforms that provide neither > Heimdal nor MIT that need SPNEGO (like Win32). See above. These are all open source projects that happily accept code donations (that meet certain quality expectations), so there's no reason to write a private copy and create more integration and interop headaches.

Here's the ASN1 for MS-SPNEGO. You can compare this to the ASN1 in RFC2478 to notice the differences. While it's possible to write code that can read both forms, it's difficult to determine when to use what. In Samba, we just have an #if block so that it's selectable at compile time. I'm not aware of an RFC2478 SPNEGO implementation of HTTP-Negotiate though so sticking with the MS-SPNEGO is probably the best idea.

> 1) under GSSAPI there is no way to negotiate NTLM since GSSAPI does not > implement NTLM. > 2) under SSPI, NTLM may be negotiated. the result is that SSPI may use its > internal implementation of NTLM, which is undesirable. OK, I may have "seen the light" ;-) Regarding issue #1, wyllys is right when he says that that is a problem that the platform gssapi can and should solve. Moreover, if servers send "WWW-Authenticate: Negotiate NTLM" then they will have a chance of failing over to using Mozilla's built-in NTLM implementation. That seems to cover #1 reasonably well as far as I'm concerned. Regarding issue #2, if we can just configure SSPI so that it does not negotiate NTLM, then I believe that we can work around issue #2. Investigating this further...

I've never actually seen IE negotiate the NTLM protocol on a packet sniff. What I always see is IE not be able to get a kerberos service ticket and then send Negotiate "Bunch of base 64'd data." If you decode the Base 64'd data you you will see that it isn't doing SPNEGO at all, instead you will see that it is doing straight NTLM. The Base64'd data decoded is actually NTLM <NTLMSTUFF>. So before we go as far as say "we need an SPNEGO mechanism to get NTLM into our cross platform Negotiate mechanism", I need to ask the question does Microsoft EVER wrap NTLM data in an SPNEGO token? Is there even a standard way of doing this? I've never seen it from IE.

At the SPENGO layer, it is certianly possible to negotiate NTLMSSP, within SPNEGO. Samba sees this all the time, on CIFS. However, my understanding is that Negotiate must simply take both, just as Mozilla sends unwrapped GSSAPI, a client may send unwrapped NTLMSSP. So, do we need SPNEGO? Well, without it we can still 'punt and see', but you will loose the ability for the server to counter-propose an authentication mechanism.

I'm attaching some code I wrote long ago which wraps the initial token in an SPNEGO. It shouldn't be that hard to implement all of the required SPNEGO protocol in a similar fashion.

The KRB5 OID used when talking to a Unix server should be the correct OID, not the broken, incorrect OID that Microsoft screwed up in their original implementation. I believe most IIS servers will now recognize both the broken OID and the correct one.

system libs are the only sane way to support this tech in 2016